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Abstract 

This article is concerned with the spectral behavior of p-dimensional linear processes in the moderately 
high-dimensional case when both dimensionality p and sample size n tend to infinity so that pjn ^ 

0. It is shown that, under an appropriate set of assumptions, the empirical spectral distributions of the 
renormalized and symmetrized sample autocovariance matrices converge almost surely to a nonrandom 
limit distribution supported on the real line. The key assumption is that the linear process is driven by a 
sequence of p-dimensional real or complex random vectors with i.i.d. entries possessing zero mean, unit 
variance and finite fourth moments, and that the p x p linear process coefficient matrices are Hermitian 
and simultaneously diagonalizable. Several relaxations of these assumptions are discussed. The results put 
forth in this paper can help facilitate inference on model parameters, model diagnostics and prediction of 
future values of the linear process. 

Keywords: Empirical spectral distribution; High-dimensional statistics; Limiting spectral distribution; 
Stieltjes transform 
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1 Introduction 

In this article, the spectral properties of a class of multivariate linear time series are studied through the bulk 

behavior of the eigenvalues of renormalized and symmetrized sample autocovariance matrices when both 

the dimension p and sample size n are large but the dimension increases at a much slower rate compared 

to the sample size, so that the dimension-to-sample size ratio p/n converges to zero. The latter asymptotic 

regime will be referred to as moderately high-dimensional scenario. Under this framework, the existence 

of limiting spectral distributions (LSD) of the matrices = y/n/p{Sr — S,-) is proved, where S,- is 

the symmetrized lag-r sample autocovariance matrix and S the lag-r population autocovariance matrix, for 
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r = 0,1,... The analysis takes into account both temporal and dimensional correlation and the LSD is 
described in terms of a kernel that is determined by the transfer fun ction of a univari ate linear time series. 
The results derived in this paper are natural extensions of the work of iBai & YinI (Il988h . who proved that the 
empirical spectral distribution of normalized sample covariance matrices based on i.i.d. observations with zero 
mean and u nit variance conver ges t o the semi-circle law under the same asymptotic regime. It also extends 
the work of ^ 


Pan & Gaol (120091) and 


Wang & Pauli (120141’) in two different ways, first, by allowing nontrivial 


temporal dependence among the observation vectors, and secondly, by describing the LSDs of renormalized 
sample autocovariance matrices of all lag orders. We need to impose a certain structural assumption on the 
linear process, namely, that its coefficient matrices are symmetric (Hermitian for complex-valued data) and 
simultaneously diagonalizable. However, various ways to relax the latter assumption are di scussed. 

The results derived in this paper can be seen as natural counterparts of the works of 


LiuetaL 


(1201 .^h . 


who proved the existence of LSDs of symmetrized sample autocovariance matrices under the same structural 
assumptions on the linear process but in the asymptotic regime p, n —)• oo such that p/n c G (0,oo). 
Jin et al.l (120141) derived similar results under the assumption of i.i.d. observations, using them for detecting 


the presence of factors in a class of dynamic factor models. Under the same asymptotic framework, the 
existence of the LSD of sample covariance matrices when the d i fferen t coo rdinates of the observed process 

Pfaffel & Schlemnl ( 2011 ) and 


are i.i.d. linear processes has been proved by 


Yaol(l2012h . 


The main results in this paper originally formed a part of the Ph.D. thesis of the firs t author dWane 


Very r ecently, we came to know through personal communication from Arup Bose that 


2014), 


Bhattachariee & Bose 


(120151) proved the existence of the LSD of symmetrized and normalized autocovariance matrices for an MA(g) 
process with fixed q, under a weaker assumption on fhe coefficienf mafrices involving existence of limits of 
averaged traces of polynomials of these matrices, where the limits satisfy certain requirements associated 
with a ^-probability space. They use free probability theory for their derivations and therefore their approach 
is very different from the one presented in this paper, which relies on the characterization of distributional 
convergence through the convergence of the corresponding Stieltjes transforms. 

The main contribution of this paper is the precise description of the bulk behavior of the eigenvalues of the 
matrices C,-. These are natural objects to study if one is interested in the understanding the fluctuations of the 
sample autocovariance matrices from their population counterparts, since the latter provide useful information 
about the various characteristics of the observed process. Under the asmyptotic regime p, n —)• oo with p/n —)• 
0, and under fairly weak regularity conditions, the symmetrized sample autocovariance matrices converge to 
the corresponding population autocovariance matrices in operator norm. However, stronger statements about 
the quality of the estimates are usually not possible without imposing further restrictions on the process. The 
results stated here provide a way to quantify the fluctuations of the estimated autocovariance matrices from 
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the population versions, and can be seen as analogous to the standard error bounds in univariate problems. 
Indeed, if the quality of estimates is assessed through the Frobenius norm of Cr, or some other measure that 
can be expressed as a linear functional of the spectral distribution of Cr, the results presented in this paper 
give a precise description about the asymptotic behavior of such a measure in terms of integrals of the LSD of 
Cr- Some specific applications of the results are discussed in Section |4] A further importance of the results 
derived here is that they form the building block of further investigations on the fluctuations of linear spectral 
statistics o f matrices such as C r, thus raising the possibility of generalizing results such as those obtained 


recently by 


Chen & Pad (12015h . 


The rest of the paper is organized as follows. Section |2] gives the main results are develops intuition. 
Section [3] discusses some specific examples to elucidate the main results. Section |4] discusses a number of 
potential applications. Sections [5]-0 are devoted to describing the key steps in the proofs of the main results. 
Further technical details are relegated to the technical Appendix. 


2 Main results 

2.1 Assumptions 

Let Z, No and N denote integers, nonnegative integers and positive integers, respectively. In the following, the 
linear process {Xt : f G Z) is studied, given by the set of equations 

OO 

Xt =AiZt_£, f G Z, (2.1) 

£=0 

where (A^: £ G No) are coefficient matrices with Aq = Ip, the p-dimensional identity matrix, and {Zt : t G 
Z) are innovations for which more specific assumptions are given below. If Ai = Op, the p-dimensional zero 
matrix, for all l> q, then one has the qth order moving average, MA(( 7 ), process 

9 

Xt = AiZt—i, f G Z. (2.2) 

r=o 

In the following results will be stated and motivated first for the MA(g) process and then extended to linear 
processes. Throughout the following set of conditions are assumed to hold. 

Assumption 2.1. The innovations {Zt : f G Z) consist of real- or complex-valued entries Zjt which are 
independent, identically distributed (iid) across time t and dimension j and satisfy 
Z1 E[Zjt] = 0, E[\Zjt\^] = 1 andE[\Zjt\^] < oo; 

Z2 In case of complex-valued innovations, the real and imaginary parts of Zjt are independent with 
EmZjt)] = E[9(Z,-0] = 0 andE[^{Zjtf] = E[Q{Zjt)^] = 1/2. 

Assumption 2.2. Suppose that 
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A1 (Af: ^ G N) are Hermitian and simultaneously diagonalizable, that is, there exists a unitary matrix U 
such that U* A^U = Ai, where A^ is a diagonal matrix with real-valued diagonal entries; 

A2 The jth diagonal entry of A^ is given by f£{aj), where cxj G for j = 1, ... ,p, where uiq is fixed, 
and {f^: i € N) are continuous functions from to M; 

A3 Asp ^ oo, the empirical distribution of{cxj : j = 1,... ,p) converges to a distribution on denoted 
by F-^; 

A4 There exist constants oq = 1 and {d £: ^ G N) such that ||/^||oo < difor all I G N; 

AS For some rg > 4, there are positive constants Lj^i such that < ^j-vi 3 — 0,..., rg. 

The conditions for j > 1 are only needed for the extension of the results for MA{q) processes to linear 
processes; see Section \2.4\ 


Liu et al 


The assumptions on the innovations {Zt: t G Z) are standard in time series and high-dimensional statis- 
tics contexts. T he assumptions on the coefficient matrices {Ai : ^ G N) are similar to the ones imposed in 


(12015h and gener a 


Pfaffel & SchlemmI (1201 ih . 


ize conditi on s ets previously es tablished in the literature, for example, the ones in 


Yaol (l2012h and 


Jin et al. 


(I2014h . 


2.2 Result for MA(g) processes 


The objective of this section is to study the spectral behavior of the the lag-r symmetrized sample autoco- 
vaiiance matrices associated with the MA(( 7 ) process {Xt : t G Z) defined in (12.21) in the moderately high 
dimensional setting 

P 

p,n ^ oo such that-)> 0. (2.3) 

n 

Extensions to the linear process (12.11) are discussed in Section |2^ below. The symmetrized sample autoco- 
vaiiance matrices are given by the equations 


where * signifies complex conjugate transposition of both vectors and matrices. It should be noted that Sg is 
simply the sample covariance matrix. Using the defining equations of the MA(( 7 ) process, one can show that 

1 / 9 -^ \ 

S. = E[S.] = 2 ( E [A£+rA| + A,A,%,] ] , T G Ng. 
e=o ' 

Since, under (12.31) . S,- is a consistent estimator for Y),-, one studies appropriately rescaled fluctuations of 
about its mean Sr- This leads to the renormalized matrices 


{XtXU + , r G Ng, (2.4) 


S. = 




2(n — r) 

^ ' t=T+ 


Cr 



T G Ng. 


(2.5) 
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To study the spectral behavior of Cr, introduce its empirical spectral distribution (BSD) Fr given by 


^ i=i 

where A,-,!, • • •, Xt,p are the eigenvalues of Cr- In the RMT literatur e, proofs of large-sampl e results about 
Fr are often based on convergence properties of Stieltjes transforms ( Bai & Silverstein . 201ol) . The Stieltjes 
transform of a distribution function F on the real line is the function 


sf: C+ ^ C+, z ^ sf(z) = f -^dF(X), 

J X- z 

where C"'' = {x + iy; x G M, y > 0} denotes the upper complex half plane. Note that sf is analytic on C"*" 
and that the distribution function F can be obtained from sp using an inversion formula. 

Let /o: —)• M be defined as /o (a) = 1 for all a € . Define fhe MA{q) fransfer function 

y(a, iz) = Y^ € [0, 2f], a G (2.6) 

£=0 

and fhe corresponding power fransfer funcfion 


V’Ca,!") = |y(a,F)|^ u G [0,2f], a G 


(2.7) 


The effecf of fhe femporal dependence on fhe specfral behavior of Cr is encoded fhr'ough fhe power fransfer 
funcfion ^/^(a, u). Keeping a fixed, if can be seen fhaf ip{sL, v) is up fo normalizafion fhe specfral densify of a 
univariafe MA(y) process wifh coefficienfs /i(a),... , /q(a). This leads fo fhe following resulf. 


Theorem 2.1. If the MA(q) process {Xt: t & If satisfies Zl, Z2 and A1-A5, then, with probability one and 
in the moderately high-dimensional setting (1231) . Fr converges in distribution to a nonrandom distribution Fr 
whose Stieltjes transform Sr is given by 


St{z) 


dF^{h) 

Z + fir{z,h) ’ 


z G C+, 


( 2 . 8 ) 


where 


firiz,a.) 


Olrisi, b)dF^(b) 

z + /3riz,h) 


z G C+, aGM^o, 


(2.9) 


and 


fRr(a, b) = — / cos^(T0)V’(a, 0)V’(b, 0)d0, a, bGM™"®. (2.10) 

2vr Jo 

Moreover, firiz^sf is the unique solution to ( 12.91) subject to the condition that it is a Stieltjes kernel, that 
is, for each a G supp(F'^), (3r{z,SL) is the Stieltjes transform of a measure on the real line with mass 
fXria, b)dF^(h). 
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Since it only differs from the spectral density of an MA(g) process by a constant, it follows that 6) is 
strictly positive for all arguments. Consequently, 3?T-(a, b) and f lRT-(a, b)(iF'^(b) are always strictly positive 
as well. The intuition for the proof of Theorem 12. II is given in the next section and will then be completed in 
Section [5] 


Remark 2.1. It is easily checked, that for an MA{q) process, the kernel lKT-(a, b) is the same for all r > 
q+1. This implies that the Stieltjes transforms Sr{z), and hence the LSDs (limiting spectral distributions) of 
are the same for t > q + 1. 


2.3 Intuition for Gaussian MA(g) processes 


Assume for now that the innovations {Zt : f G Z) are complex Gaussian, the extension to general innovations 
will be established in the Appendix. Define the p x n data matrix X = [Xi : • • • : Xn] and the p x n 
innovations matrix Z = [Zi : ■ ■ ■ : Z„]. Using the n x n lag operator matrix L = [o : ei : • • • : e^-i], where 
o and Cj denote the zero vector and the jth canonical unit vector, respectively, it follows that 

<? <? 

X = ^A,ZL^ +(2.11) 

1=0 l=l 


where Z[_q] = [^-g+i : • • • : Zq : 0 ; • • • : 0] is a p x n matrix and '^ = f 

approximated by the circulant matrix L = [e^ : ei : • • • : e„_i]. As in 


Liu et al 


^ In the next step, L is 


(120 15h . one defines fhe mafrix 


X = Yle=o A^ZL^ fhaf differs from X only in fhe firsl q columns. Lef F„ = wifh ut = 27rt/n, 


be a Fourier rofafion mafrix and A„ = diag(e 


11^1 


). Then 


L 


F A F* 


( 2 . 12 ) 


Using Ibis and noficing fhaf X and X differ by a mafrix of rank q, if can be seen fhaf as long as q small 
compared fo p, Sr = {n — r)~^XDT-X* can be approximated by Sr = {n — r)“^XDT-X*, where D,- = 
[L'^ + (L'^)*]/2 and = [L’^ + (L^)*]/2. Nofice nexf fhaf, due fo fhe assumed Gaussianify of fhe innovafions, 
fhe enfries of Z = U*ZF„ are iid copies of fhe enfries of Z, wifh U denofing fhe mafrix diagonalizing fhe 
coefficienl mafrices (A^: i G N). Define fhen S,- = U*SrU and 

Cr = ^{Sr-±r), ( 2 . 13 ) 

where X,- = E[St-] is a diagonal mafrix. If will be shown in Section [5T] fhaf fhe LSD of = U*Ct-U is 
fhe same as fhaf of Cr- 
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2.4 Extensions to linear processes 

In this section, Theorem 12. II is extended to cover linear processes as defined in (I2.1I) . To do so, the continuity 
condition A2 is strengthened to assumption A6 below. In order to approximate the linear process with MA(g) 
models of increasing order, a rate on q is imposed. 

Assumption 2.3. The following conditions are assumed to hold. 

A6 (h: I G ciT€ Lipschitz functions such that ^ — b|| foT 3., b G IR.^^ and 

^ E N, where ri < vq and tq is as in AS; 

A7 The moving average order q satisfies q = 

Analogously (12.61) and (12.71) are extended to the linear process transfer function and power transfer function 

OO 

^(a, ^/£(a)e*^'' and = \g{a.,u)\^, i/£ [0,27r], a G (2.14) 

e=o 

respectively. Then, the following result holds. 

Theorem 2.2. If the linear process {Xt: t £ Z) satisfies Zl, Z2 and A1-A7, then, the result of Theorem 12. 1 1 
is retained if (12.141) is used in place of (12.61) and (12.71) . 

The proof of Theorem l2.2l is based on a truncation argument, approximating the linear process with MA(g) 
processes of increasing order q. More delicate arguments are needed for this case as the intuitive arguments 
outlined in the previous section do not carry over to this case. Indeed conditions on the approximating MA(g) 
processes are needed that ensure that q does not grow too fast or too slow in order for the LSD of the linear 
process and its truncated version to be the same. The proof details are given in Section^below, where it turns 
out that one can choose q = 0{p^^‘^) as specified in A7. 

As a furfher generalizafion, consider fhe process (If: f G Z) fhaf is obfained from fhe linear process 
{Xt : f G Z) fhr'ough 

Yt = t G Z, (2.15) 

where if is assumed fhaf 

A8 is a square roof of fhe nonnegafive definife Hermifian mafrix B wifh ||B|| < bo < oo, and 

fhere is a nonnegafive measurable funclion qb, nof idenfically zero on supp(F'^), such fhaf for each p, 
U*BU = diag(grB(Q!i), • • • , gB{ctp)) = Ab, wifh U and Qi, • • • , ctp as defined in A1 and A2. 

Observe fhaf fhe aufocovariance mafrices of fhe process {Yt: t G Z) are given by = B^/ 2 g^gi /2 
have expecfafion 5]^ = B^/^St-B^/ 2 Assumption A8 shows fhaf fhe approximating aufocovariance mafrix 
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obtained from replacing the lag operator matrix with the corresponding circulant matrix takes on the form 


si = E^A,ZA1) { f; VAlA,ZA')* (2.16) 

^ ^ ^ £=0 ^ ^ ^ ^ £=0 ^ 

with expectation = diag(d'^^,..., d'^p) and 

1 

^T,j = ^^'^9Bio^j)cos{TUt)^p{aj,l^t), 

^ ^ t=i 

in which vt) is defined in (12.141) . Following similar arguments as in the finite and infinite order MA 

cases, it can be shown that the LSD of = y^re/p(S^ —) is the same as that of = y^n/p(S^ — 

Then, the following theorem is established. 


Theorem 2.3. If the process (Yt : f £ Z) defined in (12.151 ) satisfies Zl, Z2 and A1-A8, then, with probabil¬ 
ity one and in the moderately high-dimensional setting (|23]). FJ converges in distribution to a nonrandom 
distribution FJ whose Stieltjes transform is given by 

dF^(a) 




where 


Pt (z, a) = - y 


z + (3f{z,aL)' 
gs (a)fl'B (b)3?^ (a, h)dF-^ (b) 


G C+, 


zGC+, aGM"*°, 


(2.17) 


(2.18) 


z + fiX (z, b) 

and 3?T-(a, b) is defined in (12.101) . Moreover, {z, a) is the unique solution to ( 12.781) subject to the condition 
that it is a Stieltjes kernel, that is, for each a G supp(i^'^), (z, a) is the Stieltjes transform of a measure on 

the real line with mass f 5 'B(b)fRr (a, b)dF'^(h) whenever gsi^) > 0. 


2.5 Relaxation of commutativity condition 

The assumption of commutativity or simultaneous diagonalizability of the coefficients (assumption At) indeed 
restricts the class of linear processes for which the main result of existence and uniqueness of the limiting 
BSD applies. However, this assumption can be relaxed to a milder one in which the coefficients of the linear 
processes are only approximately Hermitian and commutative. Two such scenarios are discussed below, which 
are natural but by no means exhaustive. In both settings, it is assumed that the linear process 

OO 

Xt = t G Z, (2.19) 

e=o 

is observed with the standard assumptions Zl and Z2 on the sequence {Zt : t G Z), whereas Bq = Ip and the 
sequence (B^: £ G N) satisfies fhe conditions: 

B1 For some vq > 1, fhere are 6o = 1 and {fin'. I G N) such fhaf ||B£|| < 6^ for 7 G N and := 
< OO for j = 0 ,..., ro. 

















B2 There is a sequence of Hermitian matrices (A^: £ e N) approximating the sequence (B^: ^ £ N) and 
satisfying A1 - A6. 

In addition to B1 and B2, it is assumed that the sequence (A^: ^ £ N) satisfies one of the following conditions 
specifying the approximation property in B2: 

B3 For some 1 < /3 < 4, Sl=i ^ rank(B£ — A^) —0 under (I2.3I) . 

B4 For some 1 < /3 < 4, || B^ — A^ ||—> 0 under (12.31) . 

The importance of these conditions is discussed. First, restricting the sums involving B^ — A^ to first 
terms is sufficient in view of B1 ensuring that the process {Xt : f £ Z) can be approximated by the truncated 
process given by Xl = 'B£Zt_i with q = without changing the LSD of y^n/p{Sr — 1E[St-]). 

This can be verified by following fhe derivation in Secfion 12.41 The condition B3, on fhe ofher hand, says 
fhaf fhe coefficienl mafrices (Bf: £ £ N) can be seen as low-rank perturbations of a sequence of Hermitian 
and commufafive mafrices (A^: £ £ N). The condifion B4, which bounds fhe norms of differences befween 
fhe coefficienfs and fheir approximafions, is a bif resfricfive in fhe sense fhaf if depends on n. Presence of fhe 
factor sfnjp suggesfs fhaf fhis condifion is non-frivial essentially if n is moderafely large compared to p. 

We sfafe fhe resulf in fhe form of fhe following corollary. 

Corollary 2.1. Suppose that the linear process (Xt : f £ Z) satisfies conditions Bl, B2 and either B3 or B4, 
and let Sr denote the lag-T symmetrized sample autocovariance matrix. Then the limiting ESD of the matrix 
^yn/p(Sr — lE[Sr]) exists and its Siteltjes transform Sr(z) satisfies (I2.8I )- (I2.10I ). 

Proof of Corollary 12. Il ls given in Appendix iBl 

The condifions imposed in Corollary 12.1 l ean be used fo prove fhaf resulfs hold for processes (Xt : f £ Z) 
safisfying (12.191) and whose coefficienl mafrices are cerfain classes of symmefric (Hermitian) Toeplifz ma- 
frices. Specifically, if fhe mafrix B^ is defermined by fhe sequence (bik'- k £ Tj), satisfying fhe condifion 
sup£>i X^|fc|>m —)• 0 as m —)• oo for some s > 1, and Bl holds, fhen fhe LSDs of fhe correspond¬ 

ing normalized sample aufocovariance mafrices exisf under (12.31) provided n = 0(p^~^^^‘^). In (his case, (he 
symmefric (Hermitian) Toeplifz mafrices B^ can be approximaled by symmefric (Hermilian) circulanf malri- 
ces whose eigenvalues are precisely (he symbols associafed wifh (he sequence (bik: k £ 2i) evaluafed a( (he 
discrete Fourier frequencies 27rj /p, j = 1,... ,p. 

3 Examples 

In (his section, a number of special cases are presenfed for which (he resulfs sfafed in Secfion [2] fake on an 
easier form. 
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Example 3.1. Consider the MA( 1) process 


Xt — Zt + AZt_i, 


with A = diag(ai,--- ,ap)for aj G M (thus choosing m = 1 here). Suppose further that /i(a) = o. 
Then, the transfer function (12.61) is given by g{a,6) = 1 + ae‘® and the power transfer function (12.71) by 
'ip{a, 9) = 1 + of + 2acos(0). This yields the explicit expressions 


3?T-(a, b) 


' {1 + a^){l + b‘^) + 2ab, r = 0 . 

< (1 + a^)(l + 6^)/2 + 3a6/2, r = 1. 

^(1 + a^)(l + 6^)/2 + a6, r > 2. 


Example 3.2. Consider the special case of an MA( q) process with A^ = i = 1,... ,q, and fiiotj) = yi 
with OLj = 1/or all j = 1,... ,p. Then F'^ is a 6-function at 1. Since 

OO OO 

^(a, = 

e=o e=o 

and therefore also V'(a, o) = V'(o) do not depend on a, it follows that 

1 

3iT-(a, 1) = — / cos^{Tu){'ip{u))‘^du = 31.^, 

2vr Jo 

so that equations (12.91) and (12.81) reduce respectively to f3r{z, a) = (5t{z) = TlT-Sr{z) and 


Sr{z) = -- 


1 


Z + ‘JlrSr{z) 

For r = 0, the latter equation coincides with that for the St ieltjes transform for the case of independent 


observations with separable covariance structure discussed in mans & PauR n20lf\} . I ndeed, taking in their 
1 /2 

notation Ap = Ip and B/ = diag{g{i'i), • • • , giun)), equation (2.1) of Theorem 2.1 i mWang & PauM20l4) 
reduces to s{z) = ~[z + b 2 s{z)]~^, where 

r2TT 


62 = lim -Ti/B^) = lim - ^ Igiot)]"^ = ^ = 3 ^ 0 - 

n-^-cxD n n^oo n ZTT Jq 

Example 3.3. Consider the AR( 1) process 


t=i 


Xt = AXt-i + Zi, 

with A = diag(ai, • • • ,ap) for aj € M such that \aj\ < 1. The AR(1) process then admits the linear 
process representation Xt = 'f2’S=o With fi{a) = a^, the transfer function (12.61) is given by g(a, 9) = 

(1 — and the power transfer function (12.71 ) by 'f{a, 9) = (1 + — 2a cos 9)~^. 
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Example 3.4. Consider the causal ARMA( 1,1) process 

^{L)Xt = @{L)Zt, 

where ^{L) = I — $i(L) and @{L) = I + 0i are matrix-valued autoregressive and moving average 
polynomials in the lag operator L such that ||<l’i|| < oo and ||0i|| < oo. Then {Xt : t G Z) can be represented 
as the linear process 

Xt = A{L)Zt, 

inwhich A(L) = ~ (I —$iL)“^(I + ©iL). Assume further that and ©i are simultaneously 

diagonalizable by U, that is, U*$iU = diag(0i,..., fp) and U*©iU = diag(0i,..., Op). Let cxj = 
Oj)'^ G Assumption A3 requires that the empirical distribution of{oLi ,..., cip} converges weakly to 
a non-random distribution function defined on Note that 


U*A,U = diag(/,(ai),...,/,K)), 


with fi{oLj) = 1 and fe{oij) = {Oj + ^ for I G N. Thus, the transfer function (12.61 ) is given by 


g{a 




U = 


oo 

E. 

£=0 


h{cxi)e^^^ = 1 + V(0, + 


oo 


1=1 


t-l^i£u 


1 + OjC^ 

1 - 


and the power transfer function (12.71) is the squared modulus of the ratio on right-hand side of the last equa¬ 
tion. 


Example 3.5. Suppose that for each £ > 1, A^ is a block diagonal matrix with B (a fixed number) diagonal 
blocks such that the bth block of A^ is of the form for b = 1,... ,B, where 'f2b=iPb = P> ond 

niaxi<6<s \aih\ < oo. Suppose further that for each b, pb/p ujb as p ^ oo, where coh > Ofor all 
b. In this case, one can take ctj = b/ (m + 1) if Pfe' + 1 E 7 < X]fe'=i Pb' ond define f^ to be a function 

on [0,1] that smoothly interpolates the values {(6/ {m + 1), : 6 = 1,..., i?}. Then, Theorem \2.2\ aDDlies 


and the Stieltjes transform Sr{z) of the LSD of y'n/p{Sr — ^t) i^ given by 

B 


^r{z) = -Y. 


1 


UJb 


6=1 


2 + fiT,b{z) ’ 


gC+, 


(3.1) 


where the functions (Stieltjes transforms) fir,b{z) are determined by the system of nonlinear equations 

B 


fiT,b{z) = - ^ Wfe/ 


Rr 


, 66 ' 


where 


6'=1 


RT,bB — 


z-\- fir, b'{z:)' 


G C+, b = 1,...,B, 


1 

- / < 

271 Jo 


cos^ {T6)fb{ff)f>b'{(£)d9 


(3.2) 


(3.3) 
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with = |1 + Note that, using the notations of Theorem 12.21 PT,b{z) = (3T{z,a.) for 

a = b/{m + 1), and F'^ is the discrete distribution that associates probability ujb to the point b/{m + 1), 
for b = 1,... ,B. This example illustrates that often the precise description of ffs is not necessary in order 
for the LSDs to exist. Numerical methods, such as a fixed point method, for solving 0 . 21 ) . while ensuring that 
^WT,bi^)) > 0 whenever z G C+, are easy to implement, and can be used to compute Sr{z)for any given z. 


4 Applications 


The main result (Theorem 12.21) gives a description of the bulk behavior of the eigenvalues of the matrices 
Ct- = yJn/p{Sr — S,-) under the stated assumptions on the process and the asymptotic regime p/n ^ 0. 
Thus, this result provides a building block for further investigation of the behavior of spectral statistics of the 
same matrix. It can also be used to investigate potential departures from a hypothesized model. 

An immediate application of Theorem l2.2l is that it provides a way of calculating an error bound on S,- as 
an estimate of Hr- Indeed, if the quality of estimates is assessed through the Frobenius norm of Cr, or some 
other measure that can be expressed as a linear functional of the spectral distribution of Cr, our result gives a 
precise description about the asymptotic behavior of such a measure in terms of integrals of the LSD of Cr. 
This can be seen as analogous to the standard error bounds in univariate problems. 

One potential application is in the context of model diagnostics. Using the results for the LSD of the 
normalized symmetrized autocovariance matrices, one can check whether the residuals from a time series re¬ 
gression model have i.i.d. realizations. This can be done by graphically comparing the eigenvalue distributions 


of yJn/pS\, y^n/pS|,..., where S® is the lag-r symmetrized autocovariance matrix of the residuals obtained 
from fitting a time series regression model, with the LSDs of the renormalized autocovariances of the same 
orders corresponding to i.i.d. data. 

Further, these results can also be used to devise a formal test for the hypothesis Hq: Xi,..., Xn are 
i.i.d. with zero mean and known covariance versus Hi: Xi,... ,Xn follow a stationary linear time series 
model. If an MA(qo) process (qo can be oo) is specified, another type of test may be proposed, say, Hq : Xt is 
the given MA(qo) process (satisfying the assumptions of Theorem 12 . 21 ) . versus the alternative that X^ is a 
different process than the one specified under Hq. This can be done through the construction of a class of test 
statistics that equal the squared integrals of the differences between the ESDs of observed renormalized sample 
covariance and autocovariance matrices and the corresponding LSDs under Hq, for certain lag orders. The 
LSDs under Hq are computable by using the inversion formula of Stieltjes transforms whenever the Stieltjes 
transform of the LSDs can be computed numerically. An example of such a s etting is given by Ex ample 13.51 


The actual numerical calculations of the ESD can be done along the lines of 


Wang & Paul 


OOM). 


The test 


of whether a time series follows a given MA(qo) model, with a fixed qq, can be furfher facilitafed by making 
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use of the observation in Remark IZTl which shows that if the process is indeed MA(qo), then the LSDs of the 
renormalized lag-r symmetrized sample autocovariances will all be the same for r >90 + 1- 

Calculation of the theoretical LSD under the null model requires inversion of the corresponding Stieltjes 
transform, which is somewhat challenging due to the need for selection of the correct root, as it is necessary 
to let the imaginary part of the argument of the Stieltjes transform converge to zero. A simpler alternative is 
to compute the differences \sr^p{z) — Sr( 2 :)| between the Stieltjes transforms of the BSD and the LSDs for 
a finite, pre-specified set of z G C'*’, and then combine them through some norm (like l^o, h or I 2 ) and use 
the latter as a test statistic. The null distribution of this statistic can be simulated from a Gaussian ensemble, 
which can then be used to determine the critical values of the test. 

If the linear process {Xt: t £ Z) satisfies all fhe assumpfions of Theorem 12.21 and all fhe coefficienf 
mafrices are defermined by a finife dimensional paramefer, fhen under suifable regularify conditions, if may 
be possible fo esfimafe fhaf parameter wifh error rale Op{l/y/n) Ihrough fhe use of melhod of momenls 
or maximum likelihood (under fhe working assumplion of Gaussianily). Supposing 0 lo be fhe paramefer, 
assuming fhaf S(6*) is Iwice conlinuously differenliable and qqqqt ^t(^) has uniformly bounded norm in a 
neighborhood of fhe frue paramefer 60 , and denoting any y^-consislenl estimate by 6 , if can be shown by a 
simple application of Lemma IAT^ fhaf fhe BSD of y^n/p{ST — X)t-( 0)) converges in probabilily fo fhe same 
dislribulion as fhe LSD of ^JnJp{Sr — St-(0o)). Therefore, fhe hypolhesis testing framework described in 
fhe previous paragraphs is applicable even if fhe paramefer governing fhe system is estimated at a suitable 
precision and plugged into the expressions for the population autocovariances. 

Another interesting application is in analyzing the effects of a linear filter applied to the observed time 
series. Linear filters are commonly used to extract signals from a time series through modulating its spectral 
characteristics and also for predicting future observations. Suppose that Wt = ce^t-e where {Xt: t G 

Z) is the MA(( 7 ) process defined in Section |2A] and {c^: i £ No) a sequence of filter coefficienfs satisfying 
\ ^ Then, fhe LSDs of fhe normalized symmefrized aufocovariances of fhe tillered process 

{Wt \ t £Z) exisl and have fhe same slruclure as fhaf of fhe process {Xt : t £ Z), excepl fhaf in fhe description 
of Iheir Sfielljes Iransforms (equalions (12.81) and (12.91 )). fhe speclral densily i/'(a, v) is replaced by fhe funclion 
'0(a, c) = I YT=o v)?■ 

5 Proof of Theorem 12.11 

The concern of fhis paper is in fhe speclral properties of sample aufocovariance mafrices. Since speclral 
properties are unaffected by this change, in all of the proofs the scaling factor 1/n is preferred over l/(n — r) 
for simplicity of exposition. Throughout this section, it is assumed that the Zjt are complex-valued and the 
A^ Hermitian matrices. If the Zjt are real-valued and the real, symmetric matrices, then the arguments 
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need to be modified very slightly, as indicated in Section 11 of iLiu et al.l (1201 5h . The key arguments in the 
proof of the real valued case remain the same, since as in the complex valued case, for Gaussian entries, after 
appropriate orthogonal transformations, the data matrix can be assumed to have independent Gaussian entries 
with zero mean and a variance profile determined by the spectrum of the process. We omit the details due to 
space constraints. 

5.1 LSDs of Cr and C, 

Recall that Cr defined in (12.51) is the renormalized version of the symmetrized autocovaiiance matrix Sr- In 
this subsection it is shown that the LSDs of = U*CrU and Cr coincide, where the latter matrix is the 
renormalized version of Sr and dehned in (12.131 ). Observe that the expectation of Sr is the diagonal matrix 
^r = diag(a-^,i,..., given by 




1 "" 

- Vcos(rz/t)V'(Q:j,r't), j = 1,..., 


p. 


(5.1) 


t=i 


Now write = y/nJp{\J*SrlJ - S^), where = dmg{Ylj=o and 

define = v^(U*S^U - S^). 


We first show that = S,-, which implies equality of the ESDs of and Cr^T For each j = 1,..., p. 


u 


di) 


1 "" 

-'y'cos{TUt)'ip{aj,ut) 

71 < ^ 

t=\ 

n q 

^cos(ri/t) 

t=i e,£'=o 

q / n n \ q-r 

= i E *(“:/)*(“:/) E«"'“''+"’'''+E<!"'“'"'’‘'‘ = E*(“j)*+-(“j)- (5-2) 

" t,l'=0 \i=l t=l J 1=0 

since Y^^=i = n5o{k) for fe = 0,1,... , n — 1 where So denotes the Kronecker’s delta function. This 

proves the assertion. 

Lemma 5.1, //t/ie conditions of Theorem 12. 1 1 are satisfied, then —)• 0 almost surely under 

(12.31) . where and denote the ESDs ofC^ and Cr, respectively, and || • || denotes the sup-norm. 


Proof. Recall that Cr = s/n/p{\]*{Sr — Et-)U). Exploiting the relation between L and L, it can be shown 
that Sr = U*St-U can be written as at most 4{q + r + 1) rank-one perturbations of Sr. Hence, an application 
of the rank inequality given in Eemma IA3I implies that 


-.( 1 ) 


1 


pc.r _ F'-GI < _rank(S^ - S^) < 

p 


4(g + r + 1) 


P 


under (12.31) . which is the assertion since ' = F^^. 


(5.3) 

□ 
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Define the Stieltjes transforms s^p{z) = p“^Tr(C^ — zl)~^ and Sr,p{z) = p“^Tr(Cr — zl)~^. Repeat¬ 
edly applying Lemma IATT] to each of the rank-one perturbation matrices used in the proof of Lemma ISTTl it 
follows that, for any fixed z = w + iv € C"*", \sp.^{z) — Sp^r{z)\ < 4(g + t + \)/{vp) almost surely. It is 
therefore verified that the LSDs of and are almost surely identical. 


5.2 Deterministic equation 


In this section a set of deterministic equations is derived that is asymptotically equivalent to the set of equations 
determining the Stieltjes transform of the limiting BSD of C,-. The following decomposition will be useful 
in the proofs. Using assumptions At and A2 in combination with (12.121) and some matrix algebra, it can be 
shown that 

Sr = U*S^U = VA^V*, 


where the p x n matrix V is defined through its entries 

1 / 

Vjt =—j=g{oLj,vt)Zjt, j = 1,... ,p, f = 1,... ,n, (5.4) 

V ra 

and At- = diag(cos(T;.^i),..., cos(rr'„)). Let denote the matrix obtained by replacing the kth row of V 
with zeros, and let the n x 1 vector Vk be the kth column of the matrix V* = {vi ■. V 2 '■ ■ ■ ■ '■ Vp). Let further 
be the matrix obtained from S,- by replacing its kth diagonal entry with 0. Denote by D^, respectively, 
the matrices resulting from C,- from replacing the entries of its kth row, respectively, its kth row and kth 
column with zeros, that is. 


D 


k — 


(VfcA.V* - tr,k) 


and 


D(fc) = 




Then, 


Cr = Dfc-I-Hfc = D(fe)-I-(5.5) 

where and H(fc) = H^, -|- Wke^ with being the kth canonical unit vector of dimension p, 

hk — Wk T 


Wk = 


VkArVk and 




(5.6) 


where dr^j is defined in (15.11) . thereby ensuring that the feth entry of Wk is zero and collecting the A:th diagonal 
element of Cr in the term pk- Successively replacing rows of Cr with rows of zeros and noticing that 
Cr = C* as well as = {ekh\)* = the same arguments also yield 

p p 


Cr = Y.ekhl = Y,hkel. 


(5.7) 


k=l 


k=l 


15 






Observe next that, since its A:th row and column consist of zero entries, is an eigenvector of D(fc) with 
eigenvalue 0. If now, for 2 : G C+, R(fc)( 2 ;) = (D(fc) — zlp)~^ denotes the resolvent of D(fc), then 


^ki 


(5.8) 


that is, efc is an eigenvector of R(;j)( 2 ;) with eigenvalue —2 Let Rfc( 2 ;) = (D^ — zip) ^ be the resolvent 
of Dfc. Utilizing (15.51) and Lemma lAUl it follows that 


B.k{z)ek = R(fc)( 2 :)efc - 


'^{k){z)wkel'Ri^k){z)ek 
+ (^k^i,k){z)wk 


= —efc + -'R(^k)iz)wk, 


where the second step follows from invoking (15.81) . for the denominator part in the middle expression addi¬ 
tionally noticing that R^;.) ( 2 ) = R*^^ {z) and that e^Wk = 0 by construction. Now, all preliminary statements 
are collected that allow for a detailed study the resolvent and the Stieltjes transform of C,-. 

Lemma 5.2. Under the assumptions ofTheorem \2.1\ it follows that the Stieltjes transform Sr,p of Cr satisfies 
the equality 

for any fixed z G C"''. 

Proof Writing Ip -|- 2;(Cr — zlp)~^ = (Cr — 2;Ip)“^Cr, invoking (15.71) and Lemma lAdl implies that 

p 

Ip + z{Cr - Zlp)-^ = ^{Cr - Zlp)-^ekhl 


k=l 

P 


'^Kk{z)ek i 1 


k=l 

p 

E 


hlKk{z)ek 
1 -b hl'Rk{z)ek 


hi 


Rk{z)ekh 


(5.9) 


k—i ^ hfjRk{z)ek 

Recall that the Stieltjes transform of Cr is given by p“^Tr((CT- — zlp)~^). Therefore, taking trace on both 
sides of (15.91 ) and dividing by p leads to 


st,p{z) — 


h*kRk{z)ek 




(5.10) 


zp+ K^k{z)ek zp + hlRk{z)ek 

In order to complete the proof of the lemma, it remains to study hlRk{z)ek- Using Lemma lAUl on Rfc(2:) and 
subsequently first utilizing (15.81) and then inserting the definition of Wk given in (15.61) . it follows that 

..^{k){z)wkelR(^k){z)ek 


hlRk{z)ek = hl'R^k){z)ek - hl- 


+ elK(^k)iz)wk 


= --Kek + -h*kli(^k){z)wk 
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(5.11) 


= + ^wl'RQ^){z)wk, 

where the third step also makes use of e]^Wk = 0. Plugging (15.111) into (15.101) finishes the proof. □ 

In the next auxiliary lemma, the expected value of the Stieltjes transform of C^- is determined. More 
generally, equations for the kernel 

Pr,p{z,a) = ^Tr{{Cr - zlp)-^rr{a)) (5.12) 

are introduced, where rT-(a) = diag(lkT-(a, ctk): k = 1,... ,p) with lkT-(a, ctk) defined in (12.101) . It is a 
central object of this study and the (approximate) finite-sample companion of the Stieltjes kernel 
appearing in the statement of Theorem 12. II Its properties will be further scrutinized in Sections 1531 and 1541 


Lemma 5.3. Under the assumptions ofTheoreni \2.1\ it follows that the expected value of the Stieltjes trans¬ 
form Sr,p of Cr satisfies the equality 


E[sVp(2 )| = --E 


+ 


P ^ Z + 'K[fir,p{z,CXk)] 
for any fixed z G C"*", where the remainder term 6n converges to zero under (1231) . Moreover, 


(5.13) 


¥.[fir,p{z,a)] = — 


1 3l.r(a,Q:fc) 


+ 


P ^ Z + ¥.[l3r,p{z,OLk)] 
for any fixed z G C"*", where the remainder term 6n converges to zero under (12.31) . 


(5.14) 


Proof The proof of the lemma is given in three parts. In view of the expression for Sr,p derived in Lemma [531 
{z)wk\ is estimated first and in the second step related to $T,p{z, a). The third step is concerned with 
the estimation of remainder terms 6n and 6n- 

Step 1: For A: = 1,... ,p, let 'Flk,v = Var(ufc) = n~^diag{'f{cx.k, ut): t = 1, ... ,n) and further S-r^k = 
Ar'Sk,v^T = n~^diag{cos‘^ {Ti't)fi’{Q.k, ut): t = 1,... ,n). Define 

1 ” 

7ry(a) := -cos^{ t ut){a, ut)f {a 

^ t=i 

and observe fhat 7 ^ 7 ( 0 ) = fk(a, cxj) for all j = I,... ,p. This follows calculations similar to those leading to 
15.21 Define the matrix TT- ^ja) as the one obtained from rT-(a) by replacing its kth diagonal entry with zero. 
Observe next that the definition of Wk in (15.61) implies that it suffices to estimate the following expectation, for 
which it holds that 


77 . 77 . 

-E[vlArVln(^k){z)YkArVk] = -E[^t{A,VkV*kA,VlIl^k){z)^rk)] 
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(5.15) 


77 77/ 

= -Tr(A,5]fcA,E[V^R(fc)(z)Vfc]) = -E [TV(VfcH,,fcV^R(fc)(2))] 

= “ ['^j^r,kVj{'R(k){z))jj] = ^ [7rj(afc)(%)(^))ii] + df 

^ j^k ^ j^k 

= h [Tr(R(fc)(z)r,,fc(afc))] +4°\ 

where independence between and was used to obtain the second equality and 


4°^ = “ [{nv*Sr,kVj - 7rj(Q:fc))(R(fe)(^)) 


P 


jji ■ 


(5.16) 


ji^k 


An application of the Cauchy-Schwarz inequality to the expectation on the right-hand side of (15.161 ). subse¬ 
quently using the fact that maxj |(R(^)( 2 ))jj| < 9 '( 2 ;)“^ and squaring the resulting estimate, yields that 


[\n^*3^r,kVj - 7rj(afc)r 


1 (7^ 

Var {nv*j=.r.kVj) < 


pa( 2)2 


p^{zy ’ 


where the equality follows from recognizing that E,[nv*ST-^kVj] = ^T-,j{ock) and the inequality from observing 
that each nv*ja^^kVj is a quadratic form in the i.i.d. standard Gaussians Zji,... Zjn and has bounded variance. 
Taking the square root gives 

n 

(5.17) 




y/p^{z) 

for some constant 67 > 0. 

Step 2: Multiplying rT-(a) to both sides of the equation Ip -|- z{Ct- — zlp)~^ = C,-(Ct- — zlp)~^, then 
following the arguments that led to (15.91) . and making use of rT-(a)efc = IR(a, ak)ek gives 

p p 


r,(a) + ^r,(a)(a - zlp)-^ = J2^r{si,ock)ekhl{Cr - zlp)-^ = ^ 


^t(^) Q!fc)e/c/i^Rfc( z) 


k=l 


^ l + hl'Rk{z)ek 


Further taking trace on both sides and invoking (15.111) yields 


^T,piz,a) — ^ 


Jlr{a,ak) 


F ^ + 'Wk^{k)i^)'^k - Pk 
1 lkT-(a, Q/j) 

P^iZ + ¥.[^r,p{z, Qfc)] - efc ’ 


(5.18) 


where = K[fir,p{z, Q^)] — u)^R(fc) {z)wk + Pk- Taking expectation on the left- and right-hand side of (15.181) 
leads to equation (15.141) with the remainder term having the explicit form 

j ^ 3?T-(a, Q;fc)E[efc] ^T(a, Q!fc)efc 

+ \{z+ E[^T-,pi^^^kW{z+ ^^T,p{^,OLk)] - ek) 

It remains to show that <5^ —)• 0 under (12.31) . This will be done in the next step. 


^ — ^n,lT<5n,2' 
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Step 3: To show that 6 n 0, it suffices to verify that 6 n,i —5- 0 and 6 n ,2 0. Note that, since f3r,p{z, cx^) 

is a Stieltjes transform of a measure, 


9 (/3r,p(2;,Q:fc) ) > 


Z + E[/3r,p(2, Qfc)] > 9 + E[/3r,p(z, Qfc)] j > 9=(z) + E 

and since and w^R(;j)( 2 ;)wfc is a Stieltjes transform of a measure, 

z + E[Pr,p{z,OLk)] - Cfc = |z + WfcR(fc)(z)wfe - Pk\ > + 9'(wfcR(fe)(2;)wfc) > 9(z). 

Thus, since moreover |3lT-(a, b)| < with Li from AS, it only needs to be shown that max^ |E[efc]| ^ 0 
and maxfc E[|efc — E[efc]p] —^ 0 . 

Let R(^) = (Cr — zl)~^. Since E[r/fc] = 0, it follows from (15.151) and (15.121) that 


|E[efc]| = 


-E[Tr(R(z)r,(afc))] - iE[Tr(R(fe)(z)r,,fc(«fc))] - d® 

p p 


< 


E[Tr(R(z)r,(afc))] - E[Tr(R(fc) (z)r,(afc))] 


+ - |E[Tr(R(fc)(z){r,(afc) - r,,fc(afc)})]| + |4°^| 


P 


_ , 1,1 , , 1,2 , I ,1 

-4 +4 + 


k 


(5.19) 


where = rT-(Q:fc) — D?T-(Qfc, ctfcjefcc^ Arguments as the more general ones leading to (15.211) . imply 

that maxfc 4’^ < 6 qL‘l{p^{z))~^. Since ||R(;.)(z)|| < and Jl-rictk^f^k) is uniformly bounded, it 

follows that maxfc 4’^ < L‘l{p’^{z))~^. Together with (15.171) and (15.191) . these guarantee that max^. |E[efc] | —)■ 
0 and thus max^ |E[efc]| —)• 0. 

Observe next that, by (15.151) . 


r 1 

E [\ek - E[efc]p] = E - wlR.i^k){z)wk + -E[Tr(R(fc)(2:)r^^fc(Q:fc))] + 4°^ + Vk 
< 3E - wlTi(^k){z)wk + ^Tr(R(fc)(2:)r^^fc(Q!fc)) + % 

+ 3E -Tr(R(fc)(2:)r^,fc(Q:fc))-E -Tr(R(fc)(2:)r^,fc(Q:fc)) + 3|4' 

= 4" + 4V3m®4 


( 0)|2 


where 


4’^ < 6 E 


- w*k^(k)iz)wk + iTr(R(fc)(2;)r^,fc(Q!fc)) 


2 i 


+ 6 E[|r/fc|" 


= 6 dl’^ + 6E[|?7fc|" 


Now, maxfc E[|r/fcp] < Cp~^ for some (7 > 0 as proved in Section I dTT] It is shown in Sections [D.2I and ID3] 
that maxfc 4’^ 0 max^ 4’^ 0> respectively. Consequently, maxfcE[|efe — E[efc]p] —)• 0 and hence 

also 6 n ,2 0 . 
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Step 4: Using the expression for Sr,p(z} derived in Lemma 15^ relation (15.131) can be obtained from 
similar arguments as in Steps 1-3 of this proof. In particular, it can be shown that <5^ —0. □ 

5.3 Convergence of random part 

In this section, it is shown that, almost surely s-r,p(z) — E[st-^p( 2 ;)] —)■ 0 and (3t,p{z, a) — E[/3r,p(^, a)] —>• 0 for 
any z € C"*" when the entries of Z are i.i.d. standardized random variables with arbitrary distributions. The 
concentration inequalities on Sr,p{z) and I3t,p{z, a) are derived by using the McDiarmid’s inequality given in 
Lemma lA!2l and the proof of almost sure convergence is obtained through the use of the Borel-Cantelli lemma. 
To apply the McDiarmid inequality, treat Cr as a function of the independent rows of Z, say, zj,..., z*. Let 

Z(i) = Z - e,eJZ = Z - e,z*, j = 1,... ,p, 

where Z = [z| : • • • : z*]*. Let further be the p x n matrix obtained from the original data matrix X 
with the jth row removed, that is, 

X(„ = ^A,Z„)L'. 


£=0 
j(f) 


Define and = v^(Sr - S^), where = [L^ + (L^)*]/2. It follows then 

from the relation 

<? \ / 9 


S- = U E At(Zo) + ejz')L') D, ( ^ A,(Zo) + 

^^ i=0 ^ ^ £=0 

1 / \ 

= + -[Y^ (^jtyj£'^r^lj) + ^(jpryjta*^ + Y aj£y*iDryj£'a*i, j , 

^ £=0 £=0 £,£'=0 ^ 

where a^i = = z^L^, that 

11 11 1 

Cr = C W ^ aj,q£ + Y + E ^i£'^ji^*j£' > (5-20) 


£=0 


£=0 


£,i'=0 


making use of the notations Cj£ = (np) and = (pn) ^^'^yqAyji'. The following lemma 

will be instrumental in determining the convergence of the random part. 


Lemma 5.4. Under the assumptions of Theorem \2. 1\ it follows that 


diff,j(H) = 


Tr((C^ - ziy^U) - ^Tr((C(-^) - 


< 


3(g + l)||H|| 
p'2s{z) ’ 


where H is an arbitrary p x p Hermitian matrix with ||H|| bounded. 


Proof. First observe that YlJ £'=o^£ ^ Hermitian matrix of rank q + I and hence we can write it 

as Yl\=o^j£bj£b*ji, where each £ {—1, +1} and observe that = tijtuq — Vjiv*^^ where 
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Ujt^ = 2 + CLji) and vj^ = 2 Define the matrices Dy = Cr ^ + Yll=Q''^ji'^*j£ 

D2j = Dij - VjiVji, and notice that it then follows from (15.201) that Cr = D 2 j + Yle=o 
Therefore, 


difr,j(H) < 


H— 
p 

1 

+ - 
p 


Tr((C, - zI)-iH) - T\'((D 2 j - zI)-^H) 
1 


Tr((D2,- - ^I)~^H) - TV((Di,- - zI)-^H) 
Tr((Di, - ziy^U) - T\’((CW - zI)”^H) 

=Kji + Kj2 + Kj^. 


In the following an estimate for Kj 2 is given. For 1 < A: < g + 1, let then = D 2 j + J2e=o so that 

'T't'J+i) — application of Lemmas lA. 1 1 and lA. 31 implies that 


T}"' = r> 2 j and 


1 


9+1 


Kj2 = -Yl |Tr((Tf ^ - ^I)-^H) - TV((Tf - zI)-1h) 


1 


k=l 

9+1 


< - V 
p^i 


- zir^uiTf-^’ - zir^vjk 




Estimates for Ki and can be obtained in a similar way, leading to the bound {q + l)(p9'(2;))“^ ||H|| in 
each case. This proves the lemma. □ 


< 


(g + i)l|H| 

p'^{z) 


Lemma[5i4]gives the bound diffT-j (Ip) < 3(g + l)(p9'(z))“^ and diffT-j(r.r(a)) < 3((7 +l)(p9'(2;))“^Lf. 
Let diff(- j be defined as diffrj- with Cr replaced with C(_, where the latter matrix in turn is obtained from the 
former replacing its yth’s row z* with an independent copy (z')*. From Lemma l54l it follows then that 

1 |■I^{(C. - d)-‘) - H'{(c; - d)-')| < 

and 

i |Tt((C, - 2 l)-‘r,(a)) - Tr((c; - 2 l)-T,(a)) | < . (5.21) 

Recognizing that Sr,p{z) = p“^Tr((Cr — zl)~^) and I3r,p{z,a.) = p“^Tr((CT- — zI)“^rT-(a)) and applying 
the McDiarmid’s inequality (Lemma [A.2l) yields that, for any e > 0, 

1P(I'St,p(^) -IEKp( 2)]| > e) < 4exp 
and 

P(|/3r,p(2:,a) -E[/3r,p(2,a)]| > e) < 4 exp 

Now the Borel-Cantelli lemma implies that \sr,p{z) — E[st-,p(^;)]| —)• 0 and \(3r,p{z,a) — ¥,[pr,p{z.,Si)]\ —)• 0 
almost surely under (12.31 ). Moreover, it can be readily seen that these almost sure convergence results also 
hold for St-,p and ^r,p- 
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5.4 Existence, uniqueness and continuity of the solution 

This section provides a proof of the existence of a unique solution Sr{z) and for a £ supp(F'^) 

and z £ C"'', to the set of equations (I2.8I )- (I2.10I) . Assuming that these solutions exist, it can be shown that 
St,p{z) Sr{z) and (5r,p{z,a) (3r{z,a) for any a £ supp(F'^) and z £ C+. In view of the results 

derived in Section [531 and Lemma [531 it suffices to show that for every sequence {pj : j £ N} there exists a 
further subsequence {pj ; j £ N} such that K{/3T-,pj {z, a)) converges to a limit Pt{z, a) satisfying (I2.8I) - (I2.10I) . 
The verification is based on a diagonal subsequence argument and the Arzela-Ascoli theorem. 

Lemma 5.5. Let {pj : j £ N} denote a subsequence of the integers N and define p^^p. (z, a) = E[/3.r,pj (z, a)]. 
Then the following statements hold. 

(a) There is a further subsequence {pj: j £ N} such that pT^p.(z,a) convergences uniformly in a £ 
supp(F'^) andpointwise in z £ to a limit Pt{z, a) which is analytic in z and continuous in a; 

(b) The limit Pr{z, a) in (a) coincides with I3 t{z, a) and is the Stieltjes transform of a measure on the real 
line with mass f fR.,-(a, b)dT’'^(b) satisfying d2.9l) . 

Proof Step 1: Define 3“ = {/9T,pj(a)(') a) ^ £ supp(F'^)}. For any compacf sef K C 

|Pr,p,(a)(^;,a)| < Ll/mm^{z) = M{K). 

zGK 

Lef {ai,a 2 ,...} be an enumerafion of fhe dense subsef supp(F'^) n Q™ of supp(F'^). An applicafion 
of Lemma |A3] yields fhaf for any ai fhere exisfs a furfher subsequence {pj{ai ): j £ N} such fhaf • • • C 
{pj{ai)} C {pj(a£_i)} C • • • C {pj{ai)} such fhaf a^) converges uniformly on compacf subsets 

of C'*' to a limit denoted by Pt{z, af), which is an analytic function of 2 ; £ C"'' for each £ £ N. Choosing the 
diagonal subsequence {pj{aj): N}, it follows that 

Pr.pfafiz, ae) Pt{z, ai) {j 00 ) 

for all £ £ N uniformly on compact subsets of C"''. Note that the limit is defined on C+ X (supp(F'^) n Q™). 

Step 2: If is shown in Appendix |E] fhaf, for any fixed z £ C"'' and subsequence {pj}, {pr^pj{z,a)} 
are equiconfinuous funcfions. Since /Or,pf(a^)(^) a) converges poinfwise fo pr{z,a) on fhe dense subsef 
supp(F'^) n of supp(F'^), fhe Arzela-Ascoli fheorem fLemma lA.lOl) implies fhaf a) uni¬ 

formly converges fo a limif, a confinuous funcfion of a £ supp(F'^), fhaf coincides wifh pr{z,a) for 
a £ supp(F'^) n Thus, fhe limit pr{z, a) is now defined on C+ x supp(F'^) and is analyfic in z £ C"''. 
From (15.141) if follows fhaf fhe limif Pr{z, a) coincides wifh I3r{z, a) for a £ supp(F-^). 

Step 3: If remains to show that I3r{z,a) is the Stieltjes transform of a measure on the real line with 
mass mr{a) := f Olr(a,b)dF'^(h). This is equivalent to showing that (mT-(a))~^/3r(z, a) is the Stielt¬ 
jes transform of a Borel probability measure. The proof relies on the Lemma 15.61 stated below. From 
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the definition of /3r,p(-2,a) and the fact that rT-(a) is a positive definite matrix with bounded norm, it fol¬ 
lows that {mr^p{ai))~^l3r^p{z,aL) is the Stieltjes transform of a probability measure fip^a where mr^p{sL) = 
p“^Tr(rT-(a)). The measure /.ip,a is such that /ip,a((®, oo)) < ||r.r(a)||(mT-,p(a))“^F‘^^((x,oo)) for all x. 
Now, by the tightness of the sequence (by Lemma [5^ . it follows that {/ip,a} is a tight sequence of 

probability measures. Now, by Step 2 and the conclusion in Section 1531 it follows there is a subsequence {pi} 
such that the Stieltjes transform of (mT-,p^(a))“^/3r,p^(2, a) converges almost surely to (mr(a))“^/3T-(z, a) for 
each z e C"''. The conclusion that (mT-(a))“^/3T-(z, a) is the Stieltjes transform of a Borel probability measure 
then follows from Lemma lA.llI □ 


Lemma 5.6. Under the conditions ofTheorem \2.2\ is a tight sequence. 


It should be noted that Lemma 1531 together with Sr,p{z) Sr{z) for z G C"'', proves the existence of 
the LSD of Ct- The proof of Lemma [53] is given in Appendix ICl 

Next, we prove the uniqueness of the solutions f3{z, a) under the constraint that the solutions belong to the 
class of Stieltjes kernels that are analytic on C'*' for all a G supp(F'^). First, we verify the uniqueness of the 
solution for z G C^(no) = {z £ > vq} for sufficiently large vq > 0. At the same time, continuity 

of the solution with respect to F'^ is verified. Accordingly, let Pt{z, a) satisfy (12.91) for any a G supp(F'^). 
In view of establishing the continuous dependence of /3 t(z, a), and hence Sr{z), on F'^, on F'^ and the kernel 
Olr, suppose that there is a possibly different distribution F-^ and a possibly different kernel 31^- (but having 
the same properties as 31^-) such that ^t{z, a) satisfies 


^r{z,a) 


3irisi, b)dF^(b) 
+ j3r{z,h) 


a G 


and is a Stieltjes transform of a measure for all a G supp(F'^). Note that, by the defining equations and the 
continuity of lRT-(a, b), and ;kT-(a, b), the functions ^{z, a) and P{z, a) are continuous in a for all z G C"''. 
Also, 


/3r{z,a) - I3r{z,a) 


r 3lr{a,h){/3r{z,h) - Pr{z,h))dF^{^) 

J {z + (3r{z,h)){z + Pr{z,h)) 

_ r 3;^(a,b)d(F^(b)-F^) 

J Z + i5r{z,h) 


r {3lr{a,h) - j^^(a,b))dF^(b) 

J Z + Pr{z,h) 

(5.24) 


Define 

Wfdriz,-) Olli = j 1/3(2:,a) -^r( 2 ,a)pdF-^(a). 


(5.25) 


Then, by Cauchy-Schwarz inequality, 

|/3r(2,a) - Priz,a)\^ 
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2 


< 3 

< 3 


Jlr{si,h){(3r{z,h) - pr{z,h))dF^(^) 


{z + /3^(z, b))(2; + I3r{z, b)) 
mz,h)-^r{z,h)fdF^{h) 


+ (a) + (a) 

31^ (a, b)dF^(b) 

\z + /3r(^,b)|2|z + ^7-(z,b)|2_ 


+ r[^^(a) + rf^^(a), 

(5.26) 


where 


(a) = 3 


(3l^(a,b) - Jlr{ai,h))dF^{h) 


Z + Priz, b) 

where ||31 t- — ;Kt-||oo = supa,beK"‘o I^T(a, b) — ^^^-(a, b)|, and 


< —^ 11 3?t — Jlj 


-(2) (a) = 3 


/ 


3l^(a,b)d(F^ -F^)(b) 


z + I3r{z,h) 


< 




lyi TpJi 112 


p-^ — p 


\TVj 


where || • \\tv denotes the total variation distance. Taking vq = max{l, y/2Li}, if follows for v > vq that 

r Jil{R,h)dF^{h) 

J \z +I5r{z,h)\^\z + I3r{z,h)\^ 4 

Therefore, by (15.261) . for v > vq, 


II Pr{z,-)-Mzr) Hi < 4^(rW(a) + r(2)(a))dF^(a) 

< ^ (||3?, - 3?,||^ + 2{Lt + ||3l, - 5i,||L)l|i"^ - F^Wly'^ . (5.27) 

If F-^ = F-^, and 01^ = Ik,-, (15.271 ) and the continuity of /3T{z,a) and Pr{z,a) in a imply that I3r{z,a) = 
i3r{z,a) for z £ C^(uo) and a £ supp(F'^). Then, since both are analytic functions on C"'' for every fixed 
a £ supp(F'^), the uniqueness of the solution in z £ C+ follows. Moreover, (15.271) proves the continuous 
dependence of the solution Priz,-) on on IR,- and F-^, with respect to the topology of uniform convergence 
and that of total variation norm, respectively. From this, similar properties for Sr are easily deduced. 


6 Proof of Theorem 12.21 


In this section, the results are extended to the setting that q is not fixed, buf fends fo infinify af cerfain rate. In 
fact, q = is an appropriate choice. This rate plays a crucial role in two places of the derivations. First 

in verifying properties (such as continuity) of the solution and then in transitioning from the Gaussian to the 
non-Gaussian case. The latter situation requires the 1/4 power, while the former can be worked out under the 
weaker assumption that q = o(p^/2) it is shown here that the LSD of the truncated process is the same as that 
of the linear process almost surely. Denote then by 


S5 = 




\t=r+l 


t=T + l 


xr-rXt 


tr* 
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(6.1) 


























the symmetrized auto-covariance matrix for the truncated process Xj;’' = J2£=o t € Z. Let L(F, G) 

denote the Levy distance between distribution function F and G, defined by 

L{F, G) = inf{e > 0: F{x — e) — e < G{x) < F{x -h e) -|- e}. 

In view of Lemma IAT tI the aim is to show that 

^ 0 a.s. (6.2) 

P 

To this end, define Xt = Xt — X^^ = ^l'^t-£ and nofice fhaf 


^ n—r ^ n—T 

S cjtr \ ^/ v" v'* I 'ST v*^ \ ^/ \^tr x^tr * i v'tr 

^ t=l ^ t=l 

- n—T - n—r 

E(^‘T;2 + ^ilrX;) + - + x,+,x, 


tr*\ 
t ) 


n 


t=i 


t=i 


+ ^E(x>x:+r + Xt+rX; 

‘ t=l 

= St 1 + St,2 + St,3. 


Therefore, 


C /~ltr ||2 
r ~ IIF 


< 3 f-||ST,l - E[St,i]||| + -II St,2 - E[St,2]||f + -||Sr,3 -E[St,3]|||) • (6-3) 

\P P P J 

Hence, fo prove fhaf (16.21) holds, if suffices fo show fhaf 


^E[||ST,i - E[ST,i]|||] < oo, i = 1,2,3, 


p=i 


P- 


(6.4) 


due fo fhe Borel-Canfelli lemma. The corresponding defailed calculations are performed in Appendix IF] 


7 Extension to non-Gaussian settings 


In fhis section, if is shown fhaf Theorem 12. 1 1 and Theorem 12.31 ex fend beyond fhe G aussian setting. In order 


fo show fhis, Lindeberg’s replacemenf sfrafegy as developed in 


Chafferieel (120061) is applied fo a process 


consisting of fruncafed, cenfered and rescaled versions of fhe original innovafion enfries Ztj. To formally 
define fhis fransformafion, lef Cp > 0 be such fhaf Cp —)■ 0, —)■ oo and P(|Zii| > n^^^ep) < n 


^€p. 

The exisfence of such an Cp follows from Z1 and Z2. Lef fhen denofe fhe fruncafed 

innovations and Z^j = {Zfj — E[Zy])/(2sd(Z?^)) fhe sfandardized versions where c G {R-, 1} wifh fhe 
superscripfs R and I denoting fhe real and imaginary parts. Lef furfher Xt = Yl£=o ^£^t-£-> t £ Z, and 
define fhe aufocovariance mafrix of (Xt: t G Z) he defined by 

Ct := 7|(St-E[St]), 
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where 


S. = 


2(n — t)\ ^ 

'' > \ t=T+l 


Y. XtX*_^+ Y Xt-rY)- 




(7.1) 


The LSD of the auto -covariance matrix of Cr is the same as that of C^, since, according to 


and 


Liuetaf 


Rai & YinNl988h 


(12015h . an application of a rank inequality and Bernstein’s inequality implies that 


sup —)• 0 a.s. 

X 

For notational simplicity, the truncated, centered and rescaled variables are therefore henceforth still denoted 
by Zjt (correspondingly, Xjt) and it is assumed that they are i.i.d. with |^ii| < ]E[.Z^ii] = 0, 

E[|Ziip] = 1, the real and imaginary parts are independent with equal variance, and E[|Zii|^] = /r 4 for 
some finite constant ^ 4 . 

Consider now the process (X '^: f S Z) given by 

q 

= fez, (7.2) 

£=0 

with the innovations {Wt: f e Z) consisting of i.i.d. real- or complex-valued (not necessarily Gaussian) 
entries Wjt satisfying 

T1 E[VF^t] = 0, E[|VF^tp] = 1 and E[|lFjt|^] < C for some finite constant C > 0; 

T2 In case of complex-valued innovations, the real and imaginary parts of Wjt are independent with 
E[3?(lF,-i)] = E[9(lF,-i)] = 0 and EmWjtf] = E[3(lF,-i)2] = 1/2; 

T3 \Wjt\ < with > 0 such that Cp —^ 0 and p^^^Cp —)■ 00 ; 

T4 The Wjt are independent of the Ztj defined in Theorem 12. II 


It is assumed that the coefficient matrices (^ S N) satisfy conditions A1-A5. Define fhe lag-r auto¬ 
covariance matrix of [Xj .: f € Z) by 


s; 


2(n — r) \ ^ 

^ ^ \ t=T+l 


E 


+ E 

t=T-\-l 


-rXt 


/* 


(7.3) 


so that the corresponding renormalized lag-r auto-covariance matrix is given by 

c; = y|(s/-E[s;]) 

and the lag-r Stieltjes transform by s’^p{z) = iTr(C/ — zl)~^, z G C~'~. We denote the Stieltjes transform of 
Cr, defined in terms of the bounded (after trunctation and normalization) Zjt s, by Sr,p- Since we have proved 
the existence and uniqueness of LSD in the case where Zjt’s are i.i.d. standard Gaussian, it follows that for 
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all 2 : S C"'', Sr,p{z) converges a.s. to the Stieltjes transform of the LSD determined by (12.81) and (I2.9I) . Thus, 
proving that the results hold for non-Gaussian innovations means showing that (i) s'^p{z) — E[s!^p( 2 :)] —)• 0 
a.s. and (ii) E[st-,p(2;) — s'^p{z)] —)• 0 for all 2 : G C"'' under (12.31) . Since (15.221) has been derived without 
invoking Gaussianity of the innovatio ns, (i) follows rea dily. To show that (ii) holds requires an application of 


the Linderberg principle developed in 


Chatterieel (l2006h . This task is equivalent to verifying that the difference 

(7.4) 


E QTr(C^ - ziy^j - E - zl)-^ 

tends to zero. The arguments for (ii) to hold are provided in Appendix iGl 


A Technical lemmas 


Lemma A.l. Supposing that A is invertible and c*A ^ —1, it holds 

A-^bc*A-^ 


A + bc*)-^ = A"' - 


-1 


1 + c*A~^b 


Lemma A.2 (IMcDiarmidl (119891) Inequality). Let Ai,..., be independent random variables taking values 
in X. Suppose that f: —)• M is a function of Xi ,..., X^ satisfying, for a/Z xi,..., Xm and x', 


\f{xi,...,Xj,...,Xm) - /(Xl, . . . ,x', . . . ,Xm)| < Cj 


Then, for all e > 0, 


2 e^ 


P (|/(Ai,..., A^) - E[/(Ai,..., A^)]| > e) < 2exp - ^ 

V '^.7 = 1 S' , 


Lemma A.3 dSilverstein & Bail (Il995t) . Lemma 2.6). Let z G C"'' with v = '^{z). Let A and be n x n 
matrices with A Hermitian, and let r G C”. Then, 


|Tr ({(A - zl)-^ - (A + rr* - 2 I)"^}B) | = 


r*{A - zl)-^B{A - zl)-^r 


1 + r*(A — zl) 


< 


|B| 


Lemma A.4 (ISilverstein & Bail (Il995h . Lemma 8.10). Let A be an n x n non-random matrix and X = 
(Ai,..., A„)^ be a random vector of independent entries. Assume that E[Aj] = 0, E[|Ajp] = 1 and 
E[|Aj|^] < Then, for any integer a > 2, 

E [|A*AA - Tr(A)|“] < (i/ 2 aTV((AA*)“/^) + (i/4TV(AA*))“/^)) , 

where Ca is a constant depending on a only, and for any real function f on M, Tr(/(A*A)) = 
where Aj(A*A) is the i-th largest eigenvalue. 


Lemma A.S (IBai & SilversteinI (120101) . Theorem A.43). Let A and B be two pxp Hermitian matrices. Then, 


IlF"^ — F^W < ^rank{A — B), where || / || means sup^. |/(x)|. 
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Lemma A.6 (IBai & SilversteinI (l201flh . Theorem A.44). Let A and B be two p x n complex matrices with 
ESD’s and F^. Then, 

||_pAA* _ 


1 


< -rankfA — B). 

P 

More generally, if C and D are Hemiitian matrices of orders p x p and n X n respectively, then, 

ll^c+ADA* _ ^C+BDB‘|| < lrank(A - B). 

P 


Lemma A.7 (IBai & SilversteinI (120100 . Corollary A.40). Let A and B be two n x n normal matrices with 


ESD’s F^ and F®. Then, L^{F^,F^) < n ^Tr((A — B)(A — B)*), where L{F,G) denotes the Levy 
distance between distribution functions F and G. 


Lemma A.8 (IBai & SilversteinI (|201(J) . Theorem A.45). Let A and B be two pxp Hermitian matrices. Then, 
L{F^,F^) < ||A-B||. 


Lemma A.9 (iGeronimo & Hilll (120030 . Lemma 3). Let T be a family of functions analytic in an open con¬ 


nected set D. If for each compact set K in D there is a constant M (K) such that 


\f{z)\ < M{K) for all / G T and z £ K, 


(A.l) 


then every sequence in T has a subsequence that converges uniformly on compact subsets ofT) to a function 
analytic in T>. 

Lemma A. 10 (Arzela-Ascoli), A sequence of continuous functions on a compact support converges uniformly 
to a continuous function if they are equicontinuous and converge pointwise on a dense subset of the support. 


Lemma A.ll ( Liu et al 


(120150 . Lemma S.13). Suppose that {Pn) is a tight sequence of Borel probabil¬ 


ity measures with corresponding Stieltjes transforms {sn{z)). If Sn{z) s{z) for ail z G C"*", then 
lim^^oo ius(iu) = — 1 and thus s{z) is a Stieltjes transform of a Borel probability measure. 


Lemma A.12 (IGeronimo & Hilll (120030 . Theorem 1). Suppose that {Pn) are real Borel probability measures 
(with mass 1) with corresponding Stieltjes transforms {sn{z)). If lim^^oo Sn{z) = s{z)forall z with ^{z) > 
0, then there exists a Borel probability measure P with Stieltjes transform sp = s if and only if 


lim ivs{w) = —1 

V^OO 


(A.2) 


in which case P„ ^ P in distribution. 
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B Proof of Corollary 12.11 


In view of Lemma I a 3I and a truncation argument analogous to that in Section 0 without loss of generality, 
attention can be restricted to the matrix 

qp-T 


c? = 


n / - 




£=0 


where Sf = ^X^D,(X^)* with D, = i(L- + (L")^), X® = and qp = 

since /3 G [0,4). Define X"^ = A^ZL^ and = ^Jn/p{S^ — Y^'^i=q where S:^ = 

^^^X'^Dt-(X"^)*. It suffices to show that the distance between the ESDs of and converge to zero 
almost surely under conditions B3 or B4 and Bl, B2, A1-A5. 

First, to prove the result under condition B3, by Lemma Ia31 it suffices to show that 

-rank (Cf - C^) ^ 0 a.s. 


P 


In this direction, first note that. 


^-rank(Sf - S^) <-rank(X^ - X^) 


p 


P 


^ qp 2 

< - ^rank(B£ - A^) < - ^ rank(B^ - A^) 


P ‘ 


0 , 


!=0 


1=0 


where the last condition is by B3. Also, 

qp-T 

p \ 2 


qp-T 


-rank ^ ^ (B,B,%, + B,+.B|) - A,A,+. 


i=0 

qp-T 


e=o 


Combining the last two displays, (IB. Ill follows. 

Now, to prove the result under B4, by Lemma lA^ it suffices to show that 

IIC^ - C^ll ^ 0 a.s. 


(B.l) 


2 ■ Qp r ^ 

< -rank( ^ (B^ - ^ A(,{B^+r - A^+r)*) < - E i‘ank(B£ - A^) 0. 

^ £=0 1=0 ^ 1=0 


(B.2) 


where || • [ji? denotes the Frobenius norm. As a first step, note that since ||L|| < 1, where L is the lag operator, 

i/n 


-||Sf-Sf||< 

P Vp{n - r) 


max|||X^||,||X"^imiX^ -X'^l 


<- 




qp 


qp 


<- 


Vp{n-T) 
n ,, 1 


jZf J](||B,|| + ||A,||)^||B,-A, 


i=0 
/ oo 


e=o 


n — T n 


zz’ii E^' + E 




^r=o 1=0 


rpi/«i 


E IIB' 


— A.tf|| —y 0 a.s. 


£=0 
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Here the last line follows from assumptions A4, AS, Bl, the fact that ||^ZZ*|| < 1 + e a.s. for large n, for 
any given e > 0, and assumption B4. Next, 

— qp-T qp-T 

-II- (B,B,V + B,+.B|) - ^ A,A,+.11 

P ^ £=0 1=0 


< (max ||B/|| + max \\Af 
£ £ 




||B£ — A^ll —)• 0, 


?=0 


again by A4, AS Bl and B4. Combining the last two displays, (IB.2I) is obtained. 


C Proof of Lemma 15.61 


In view of Lemma l5d1 and the truncation arguments in Sections and |7J it suffices to show that is 

a tight sequence, where C. = y/n/p{Sr — E[S.]) and S. = ^XD.X* where X = A^ZL^ and 

D. = + L~^), with Qp = and the Ztj satisfying Zl, Z2 and \Ztj I < where Cp > 0 is such 

that Cp —)• 0 and p^/^Cp —)■ oo. This is established by showing that iTr(C^) = f x^dF^'^ is bounded almost 
surely, which in turn is shown by verifying that p“^E[Tr(C^)] is bounded from above and 

2n 


P>1 


SE[Q^'(C?)-iE[lV{C?)] 


< OO, 


the result whereby follows from the Borel-Cantelli lemma. 

Define, = ZE^^'Z* - Tr(E^^')Ip and = A^A^/. Observe that 

Tr(E''') = lTf(L^-^'+-) + iTr(L'-''-") = ^{6oii' - i - t) + 5o{£ - - r)). 


Then, 


Qp Qp 


Qp Qp 


c. = 


- Y A£^(ZE^i^2z* _ e[ZE^i^2z*])a^^ 

£l=0 l 2=0 


Y E 


V ^£^= 01^=0 


£2 


and hence 


. ^ Qp Qp Qp Qp 

E E E E 'I''(U'''-Afe+ 3 U'>'<A,,A, 

^ ^ £ 1 = 0 ^ 2 = 04 = 04=0 


4. 


1 

np^ 


qp qp qp qp p p p p 


EEEEEEEE u';&Gggug'‘G 


£ 4^1 

*4*1 ■ 


(C.l) 


(C.2) 


(C.3) 


£1=0 £2=0 £3=0 £4=0 *1=1 *2=1 *3=1 *4=1 

Let the i-th row of Z, written as an 1 x n vector, be denoted by z* = {Zn , • • • , Zin) = (zf + i(zf where 
zf = (5R(Zii), ... , ^(Zin)) and zf = ( 9 (^* 1 ), • • •, 9 ‘(^*n))- Thus, z* = zf - izf, and hence 

=z*E^'z, - TV(E^'>o(i - i') 
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- lTr(E''')<5o(i - i')) + ((^f - iTr(E^^')5o(i - i') 

-i(zffE'''zf, + i(z[fE'''z« 


(C.4) 


^ii' ' ^ii' ' ^Vn' ’ 


(C.5) 


say. Notice that x^f, = y^-. Further, expectation of each of the terms in the last line of (1C.41 ) is zero, which 
follows from 


E[zf(zfr]=E[zazi)^] = -I„ 


E[z, 




= 0 


pxp 


(C.6) 


and the independence of Zj. It can then deduced that 


IE[uSg<t] = 0 and E[Uggugg] = Tr(E^i^^E^3^4)^ ^ 

while 

n 

E[Ug^2Ug^"] = Tr(E^i^2E^3^4) + - 2) 

f=i 

where /i 4 = ElZnl"^. Note also that Yl'j=i except when either \ii—i 2 \ = t or\(.‘i — ti\ = r. 

Now |E[Tr(C^)] can be computed. Recalling that that E[Ug/] = 0, and using (1C.31) and (1C.61) . and 
independence of Zj, it follows that 

-E[Tr(C2)] 

P 

I Qp Qp Qp ^p P P 

“^^EEEEEE 

^ 1=0 £ 2=0 ^ 3=0 ^ 4=0 ii=l 127^^1 
Qp Qp Qp Qp P 

E EE|uS'’u'i'‘]G'y>G'‘''. 

£ 1=0 £ 2=0 4=0 £ 4=0 j=i 

From this, (IC.2I) . the calculations above and recalling A4 and AS, it follows that |E[Tr(C^)] is bounded from 
above. 

A more involved calculation, involving the computation of E[Ugg3uggugg®Ugg], where the indices 
ik are paired, with several applications of Lemma lA.4l (for dealing with terms where the same index ik appears 
at least six times), proves that E[(4Tr(C^) — 4E(Tr(C^)))^] < M/p^ for some finite constant M, which 
implies (1C.II) and completes the proof. Detailed calculations are omitted due to space constraints. 



D Auxiliary results for Section 15.21 

D. 1 Estimation of E [ 1 77 ^ | ] 


In order to verify E[|? 7 fc|] —)• 0, it suffices fo show fhaf E[|r 7 fcp] —)• 0, since (E[|? 7 fc|])^ < E[|77fcp]. Indeed, 


E[|%|2] =E 


-{vlAkVk - ar,k) 
P 
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= —E 
np 


t=i 


= —E 
np 

= —E 
np 


'^cos{Tnt)'ip{ak,nt)\Zkt\‘^ -cos{ t nt)'ijj{ak,nt) 

L t=i 

^cos(rt'i)V’(Q:fc,t't) j (l^fciP - 1) 

t=i ^ 

n n 

EE cos{Ti't)cos{Ti'ti)'il){cxk,vt)'il){cxk,vt'){\Zkt\^ - l){\Zkt'\‘^ - 1) 

t=l t'=l 

— Vcos^(rt't)T/>^(Q:fc,i/t)E[|ZHp - l]^ 
np 
^ t=i 

1 ^ 

^cos^(rz/t)V’^(Q;fc,z/t) 


t=i 
Pi-1 


np 


t=i 


< 


C 

P 


for some constant C > 0, where pi = £’[|Zfci|^]. 

D.2 Estimation of maxfc (fff 

Let Jfc = iTr(R(fc)( 2 ;)r^^(Q:fc)) — E[p“^Tr(R(^)(z)r”^(Q:fc))]. It follows from (15.231 ). that, for all v > 0, 

E(|JfcP >y) < 4exp 

for some cq > 0. Thus, setting cq = CQpq~‘^, 

lE[|JfcP] = I yPdJfcP > y)dy 

Aq^ 


copy\ 

J’ 


4 f°° 2 4 

< ^ / Coyexp(-coy)d?/ = T2r(2) = 
Cn Jo Cq 


2 9 ’ 

CqP^ 


-0 0 '^0 
where r(-) is the Gamma function. The right-hand side goes to zero if p oo. This continues to hold even 


, 2,2 


0 under (12.31) . as 


if the the MA order q grows at a rate satisfying q^ = o{p). Consequently, max^ 
required. 

D.3 Estimation of max^ 

Throughout this subsection the following fact is repeatedly used: 

OO OO OO 

IV'(a, vt)\= ^ <^ae^an < lI, 

e,i'=o e=o e=o 

which holds, since |/£(a)| < || A^H < and a£ < Li by assumptions A4 and AS. 

Let Wk = ^/njp\ k^rVk and recall that = [ui,..., Ufc_i, 0, Vk+i-, ■■■,Vp] and Vk = n~^/‘^SkZk, 
where Sfc = diag( 5 (Q:fc, vt) \ f = 1,..., n) and Zk is the n x 1 column vector with entries Zkt- Thus, 

71 

w*kR.[k){z)wk = -UfcA^V^R(fc)(z)VfcA^Ufc 
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= ^Tv{R^k)iz)VkArVkV*^Arn) 

= ^Tr (R(fc)(z)VfcA,GfcA,V^) + , 


?(i) 


where 


= ^Tr {A^VlR(^k)iz)^^kAr{nvkvl - Gfc)) 

= —'^^(R(^k}iz))jjVjAT-\JkArVj, 

^ j^k 


(D.l) 


and Ufc = nvkvl - Gk and Gfc = nAlk^v = nE[vkvl] = diag(T/'(Q:fc, t'*): t = 1, ..., n). Define the {j,f )th 
element of Qk = VfcA^GfcA^V^ as Qk{j,j') = v*ArGkArVj/, and notice that Qk{k, k) = 0. Then, 

Rk = w*kK(^k){z)wk - iTr(R(fc)(z)r”^.(Q:fc)) 

= i-IV (R,i,(2) {Qt - r;>j(aO}) + 4" 

= Rf + 4'>. (D2) 

where 


Rf^ = ^Tv{K^k){z){Qk-rik{c^k)}) 


1 


^ ^X ^ik)i^))jj{'^i A^GkAT-Vj 'y^^k{^j))~k ^ ^ (^(k){^))j'j{''^j ArGkAT-Vj/) 

^ j^k P j^j'^k 


_ P^d I p2,2 

— Rk + Rk 


(D.3) 


with 


- i^ik)i^)) a (-'^cos‘^{Ti^t)i’{ak,i^t)i’{aj,i't){\Zjt\‘^ - 1)Y 

P j^k t=i ^ 

2 / ^ _“ \ 


r2,2 _ 

P^k — 


j¥^j'¥^k 

Using independence of Zji ,..., Zjn, ]E[Zjt] = 0 and E[|Zjip] = 1, it follows that 

1 ” G 

E04V] = - If] < ^ 

P i^k t=i '' '' 


C 

np^ 


(D.4) 


where CT-jfc(; 2 ) = cos^{Tv)'ip{oLk,v)'4>{ocj, 1 ^) and the inequality results from ||R(fc)( 2 ;)|| < 9(-s) Similarly, 

n 

11 =;x;2 E l(R(A:)(^))i4PE|Crd'7fc(^*)l'E[I^Tt|']IE[l^it^ 


n^p^ 


j¥^j'¥^k 


t=i 
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^ ^ _ _ 

+ „2„2 CTjijk{h't)cT,jj'k{t^t)^[\Zjit\ ]]E[|Zjt| ] 

^ jT^r^k t=i 


< 


c 1 


n’ 


(D.5) 


where Crj>jk{k') = cos^(rz/)'i/’(Q:fc, vt)g{cxji, v)g{oLj, u), making also use of the fact that E(Zji)^ = 1 since 
the real and imaginary parts of Zjt are independent A^(0,1/2) random variables. 

Next, consider 


^ i^k 

H 2 'y y (R'(A:) (r^(fc) ^rU/j AT-lJj'Uj/A t-XJ/j AT-'Uj/j 

+ X] (f^(fc)(^))ii(^(fc)(^))iY®^[Tr(SjA^UfcA^gj)Tr(g*,A^UfcA^gj/)]. (D.6) 

Define = g* At-U^A^ g^ and the (s, i)th element of Bj^ by 6jfc,r(S) f) for 1 < s, f < n. Observe that 
^*jk,T = ^ 3 k,T- Also, 


Tr(B,fc,,) = 4*g^A,g,g*A,gfeZfc - Tr(g^A,g,g*A,gfc) 

n 

= '^Cr,jk{k't){\Zkt\^ - 1) 
t=l 
n 

— 'y ^ bjk^ri^j t). 

t=l 


Thus, for j / f / k. 


E[Tr(g*A,UfcA,g,)Tr(g*,A,UfeA,g,v)] 

= E[Tr(Bjfc^^)Tr(Bj/fc^^)] 


= E 


t=l t'=l 

^Cryfc(i^t)crj'fc(r't)E[(|Zfci|^ - 1)^] < Cn 


t=i 


(D.7) 


for some C > 0 uniformly in j,j', k using the independence of the Zki ,..., Zk^n and that E[|Zfctp] = 1 for 
all t. Utilizing the same arguments, for j / k, 


E[{Z;Bjk,rZjf\Vk] 


34 













= E 




t=l t’ = l 
n n 


Ufc 


t=l t'=l 

n n n 

t=l t=l v = l 

n 

- 1) E i) + Tr((Bjfc,r)^), 


(D.8) 


t=i 


where = E[|Zj 4 1'^], noting that the last step makes use of BE ^ = ^jk,T- Now, 


E 


Y.{bjkAt,t)? =Y,4,jkAtH{\ZkA - 1)1 

-I t=l 

n 


t=i 


(D.9) 


t=i 


for some (7 > 0 uniformly in j and k. 

Finally, define Fj^^t = A^-S^Sj AtS/c- Observe that Fjk^r is a diagonal matrix with (f,f)th element 
Cr,jkAt) for t = 1,..., n. Thus, 

E[Tr((B,fc,^)2)] = E[Tr(UfcA,g,g*A,UfcA,g,g*A,)] 


= E 


= E 


= E 


f [AkZkZlTk - %Tk)^r9A*AA%ZkZ*,9*k - %Tk)^r%% 

z*^S*kArSj9*ArSkZky -Tr [(g^A,g,g*A,gfc) 
n \ 2-1 n 

^ A rJkAt)\Zkt\ j ~ ^ ^ C^,ikAt) 


t=l 


t=l 


^2(^l,jkAt)F[\ZkA - 1] + E (^TjkAt)cT,jkAt')F-[\ZkA]F[\Zkt'A 

t=i t^t' 

n y n ^2 

(/i4 - 1) E ^rJkAt) + ( E Cr,jfc(t't) 


t=l 


t=l 


< cA, 


(D.IO) 


for some (7 > 0 uniformly in j and k. 
Combining (ID.6I) - (ID. 101) leads to 




p n 

- 2 ,: 

k 


(D.ll) 


for some constants Ci, (72 > 0 uniformly in k. Thus, noticing that = E[|i?fcp] and combining (ID.2I) - (ID.5I) 
and (ID.l II) . it follows that 


maxd?’^ < 


1 fC[ , C' 




— + — 


\ p n 
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for some , C 2 > 0, so that max^ is asymptotically negligible. 

E Equicontinuity of (5t^p{z^ a) 


In this subsection, it is verified that I3r^p{z, a) is uniformly equicontinuous in a. Observe that 

3?^(ai,b) - 3?^(a2,b) = — / cos^(rz^)i/>(b, i/)(i/>(ai, -'0(a2, 

1 

= ^/ Tpih, u) {ip {a.1, u) - ^I;{a2,i^))di^ 

1 

+ — / cos(2ri/)V'(b, z/)(V'(ai, z/) -'0(a2, 

Jo 

= K^ + K^. 


Recall that, by A6, for each £ > 1, is a Lipschitz function satisfying, for any ai, a 2 G 

l/Kai) - /£(a2)| < Cr^llai - a2|| 
for some C > 0 and some integer ro > 4 as in AS. Therefore, 

(■2-k/ °° °° 


(E.l) 


1 f'^'K/ \ / \ 

''^l2=0i'^=0 ' 

/ 2 - 7 ^- 00 CXD 00 00 

X] Z] X] /£i(ai)[/£;(ai)-/£;(a2)]/£2(b)/£/(b)e‘(^i-^'i+^2-£^);^^^ 

i' —n Z'o=n —n 


1 

dvr 


£1=0 £^=0 £2=0 l'^=0 

(' 2 'k 00 00 od 00 


+ 


^ / X] X] X] X] /n(^2)[/£i(ai) - /£i(a2)]/f2(b)/^/(b)e'( 

*^0 —n <9/ n D n ot n 




£1=0 £'=0 £2=0£^=0 

00 00 00 

E EE* i(ai)[/£i-m(ai) - /£i-m(ai)]/£' 2 (b)/^^_^(b) 


l\=ml'^=m m =0 
00 00 00 


+ X] X] X]/£i-m(ai)[/£i(ai)-/£i(ai)]/^/^(b)/^/^_^(b) 


£l=m l'^=m m =0 


and 


X 


-j rZTT / 00 00 

^5 = ^ / cos(2tz/)( Y [Ai(ai)/£'(ai) - /£i(a2)/£/(a2)]e‘(^i“^'i)'^ 

^ ^£i=or;=o 2 

00 00 \ 

£2=0£^=o 

"I /*^ 7 T 00 00 00 00 

— / cos( 2 tz/) E E E E /^i(ai)[/n(ai) - /£;(a 2 )]/£ 2 (b)/f/ 

£i=0£;=0£2=0£^=0 
^ /*27r 00 00 00 00 

^ 4 ^ / E E E E /n(^ 2 )[/£i(ai) - / 4 (a 2 )]/£ 2 (b)/^/ 

•'0 <1, —n/I'—n i’'—n 


£1=0 £'^=o£2=0£^=0 
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= E E EM ^l) [/^i—m—r (S-l) /fi—m— 0^) fl'^—m—Ti^) 

£l=m+T £'^=m+T m=0 
oo oo oo 

+ E E E*.— r(ai)[/£i(ai) - /£i(ai)]/^^(b)/^^_^_^(b). 

£l=m+T £^=m+T m=0 

Therefore, by the Lipschitz properties of the f/s, 

OO OO OO 

l^4l < X] ^ X] -/£i-m(ai)||/^/(b)||/^/^_^(b)| 

£l=m £^=m m=0 

OO OO OO 

£i=mt^=mni=0 
oo oo oo 

12 12\^^-^\'’°\fhi^^)\\f£'2i^)\\f£'^-mi^)\\\^l-^2\\ 

ii=7n i'^=m m=0 
oo oo oo 

+ CE E El«it“IA.-’»(‘>i)ll4('=)ll4-™(‘')ll|a,-ajil 

£l=m £'^=m m=0 

= K 41 + ftr42. 


Using A4 and AS, one obtains the bound 

000000 

K42 = C^J2Y^ |(£i -m) - (4 -H +4r°l/u-m(ai)||/£'2_m(b)||/£^(b)|||ai -aall 
£1=0 £'^=0 m=0 

000000 

< C'3'’°"^||ai - a2|| 12 12 12 [ 1^1 “ m|’'°d£i_md£2/-md£^ + I4 “ ^r°%-ma£i-mae'^ 

£1=0 £'2=0 m=0 
+ 1^21 ^^£'^£\—m^£'^—m, 

< CL^L^p+i ||ai — a2||. 

Similarly, it can be shown that K 41 < CL^L^o+illai — a 2 ||, thus implying K 4 < CL^Lro+i||ai — a 2 
Following the same steps yields also that 

< CL\LrQ+i\\a.i -a2|| 


and hence, for some constant Cq > 0, and for all b G ]R"*°, 

|3?^(ai,b) - 3J(a2,b)| < CqUiL^o+i|| ai - a 2 ||. 

This shows that lRT-(a, b) is Lipschitz with with respect each variable with a bounded Lipschitz constant Cq. 
Thus, the equiconinuity of I3r,p{z, a), for any z with 9(^;) = n > 0, follows from 

\f3r,p{z,ai) - / 3 r,piz, 8 L 2 )\ < ||(C^ - 2:/)(T^(ai) - T^(a 2 ))|| 

< — ai - a2 , 

V 

observing that rT-(a) = diag(3iT-(a, otj): j = 1 ,... ,p). 
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F Auxiliary results for Section 


As a first step, an inequality is derived for bounding discrete convolutions of the sequence (a^: £ € N) that 
appears in assumptions A4 and AS. 


Lemma F.l. Let {ai ^ be as in A4 and AS and tq be as in AS. Then, for r < tq, 


OO 

e=k 


< 


2r+l 

1 + |u| 


e=k 




i=k 


(F.l) 


for any A: G Nq and u G Z. 


Proof Fix /c G No and tt G Z. Plancherel’s identity and integration by parts (r times) yields that 


where 


and, for r > 0, 


Since 


t=k 








^ 1 f'ZlT ^ 

Y.didi+u = —f 

l=k 

['■ 1 r 
2 -k Jq 

OO 2 oc 






£=k 


OO OO 


ai^ai^ 


h=q+l A =<?+1 


’/’[«(«) = r E E - l2re'^‘'-‘-^'>ae,ae,. 


ti=k t2=k 


sup j^';:'(e)i < 2'-i E Em+' 2)56 o& < 2q E''5() (E5')' 


the assertion of the lemma follows. 


t=k 


i=k 


(F.2) 


□ 


F.l Bounding E[||S,,i -E[S,,i] III] and E[||S ,,2 - E[S,, 2 ]|||] 

Notice first that, for i = 1,2,3, E[||Sr,i — E[Sr,i]|||] = E[||Sr,i|||] — ||E[Sr,i] |||. Moreover E[St-,i] = 
{2n)-^E[Y:UiXtXtf,* + Xr+rm and 

ll®'[®'r,i]|lF ~ ~ Tr(Am/_i_T-Am+T-AmA„^/) = 0, 

m'=max{—m=max{—T,g+1} 

I|E[S.,2]||| = (l - 0 Tr(A^,_,A™_,A™A.^0- 

m'=max{T,5+l} m=max{T,q+l} 
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Since the arguments for bounding E[|| St -,2 |||’] are similar, the focus is here on bounding E[||St-,i |||,]. The key 
decomposition is 


1 

2n^ 


n—Tn—T 


El IISx,iIIf] =£2 E E + t-2 E E 


t=l s=l i=0 £f=0 ra=q-\-l m'=q^l 
^ n—T n—T q q 00 00 

+ 


i=l s=l 

n—T n—T q q 00 


1 

2n^ 


n—Tn—T 


t=l s=l 


,-A.m Am' Zt. 


211 ? 

=Qi + Q 2 


t=l s=l r=0 £'=0 m=g+l m'=5-|-l 


By independence of the Zij, the summands in Qi and Q 2 are non-zero only if the indices of the Zj pair up, 
giving four types of pairs that contribute to the summands in Qi, namely 

1. t + T — i = s + T — i'^s — m = t — m', that is, t = s + £ — f = s + m' — m and ^ m + t\ 

2. t + T — £ = s —m^s + T — £' = t — m', that is, f = s + i — m — T = s + m! — £' + t and ^ m + T\ 

3. t-\-T — i = t — m'^s —m = s + T — £'■, 

4. t + T — l = s + T — l' = s — m = t — m', that is, t = s + I — £' = s + m' — m and £' = m + r. 


The corresponding terms are labeled Ai,i, Ai, 2 , Ai ,3 and Ai, 4 . The individual contributions of these terms 
can be given as follows. First, setting ui =£ — £' = m' — m 

n-T q cxD mm{n-T-s,q-e'} 

^u = ^EE E E Tr( A^' A£'_|_tjj^ )Ti ( Am Am-i-ui ) • 

s=l £'=0 m=g+l ui=max{l—s,—£',q+l—m} 
m^£'—T 

Second, setting U 2 = i — m — t = m' — £' + t, 

^ n-T q 00 mm{n-T-s,q-{m+T)} 

^1-2 E E E E 

s=l £'=0 m= 9 +l u 2 =max{l—s,—(m+T),q+l+T—£'} 
m^£'—T 

X Tr(Am+n2-l-'rAmA£/A£/_|_Tf2_T-)E[.Z^_^^]^]E[Z’^_l_.j__£/^]^]. 


Third, 


^ n—T n—T q—T q—T 

E E E E -\-T-^m' -^m-\-T 

s = l t=l m=max{ —TjQ+l} m'=max{-T,q + l} 

^ n—T n—T 9.~'^ 

EE E E 

s=l t=l m=max{—Tj^+l} m'=max{— 


1 


n—Tn—T 


Q-'T 


~\~ 2,^2 ^ ^ ^ ^ ^ ^ Tr(A 777 ,-|_£_ 5 -(_- 7 -Att^-I-t-A 77 Xs) 

s=l t=l m=max{—r,g+l} 
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+ kI% 


Observe that coincides with (1/2 )E[||S't-^i |||,]. Finally, 


^ n—T q—T 

^h4 = 15-2 


mm{n—T—s,q—{m+T)} 


2 n 2 


E 


Tr ( ) Tr (A 772 ) 


s—l m=max{—r,g+l} u=max{l—s,g+l—m, —(m+r)} 

P 

“1“ ^ ■^m+T+u) kk kk [ | ^s—m,k \ ] l) 

k=l 

These quantities are bounded using the basic bound HA^H < a^. In the following, let = 0 for ^ < 0 and 
denote by C a generic positive constant. For r > 0, 


- n—r q oo 

I^uIS5;3EE E 


mm{n—T—s^q—i'} 


E 


2 - - - - 

P CLi!CLiij^^^CLfji(lnn+u\ 


s=l l'=0 m^q +1 u\=ma.y.{l—s,—l',q+l—m) 


< 


p 


2 00 / 00 


‘ 2 n 

<2^0 


E (E ai'^e'+u 

1 


u=—oo ^^=0 

9 CXD 

+ir 

n 


E 


u=—oo 

9 / OO 

E 

n 


(1 + |tt|'’0)2 


OO \ 

^ ^ ^m^m-\-u I 

m=q+l ' 

00 \ / 00 

m=q+l 


e=o 


?=0 


rrf°am 


m=g+l 


' m=g+l 


rrf°am 


^ ^ I ) 

m=g+l 


where the third inequality follows from (IF. II) . Next, 

n—T q 00 

-2v? 


^ n—T q 00 


min{n—r—s,g—(m+r)} 

E 


P^m+U2+T^m^i' +U2—'J 


5—1 £'=0 m=g+i ti 2 =max{l—s,—(m+r),g'+H-r—£'} 
m^i'—T 


00 / 00 


s|;E E 

w=—OO ^£'=0 


(Z£/ r 


^C^^l^ro+ 1 " 




n 


m=q-\-l 


^ ^ (3-77xCl777,-(_'^-(_'7- 

m=g'+l 
00 

m=Q+l 


Also, 


- n—Tn—T q—T 

^^Si^iEE E 


^ F - _ _ _ 

^ ^ dm+T+uOim+TO'mO^m+u 


F®m+t—s+rOm+T^mOm+i—s 


s—l 1—1 r?i=max{—T, 5 +l} 
00 CX) 


< 


2n 

IL 

' 2n 


u=—oo m=q+l 
00/00 


E E 

w=—00 ^ m=q+l 


^m+T+u^m+T 


m=q+l 
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Finally, 



^ n—T q—T 


min{n—r—5,0-—(m+r)} 

2n2 ^ ^ E -|- P^EI^I^Z”!!! j l) ] ftm+r+ti^m+rQ'mtim+w 

5=1 m=max{—T,g+1} w=max{l—S jQ'+I— m,—(m+r)} 

2 oo oo 

e O^m+T+uO^m+rO^m^m+w 

TT' 

u=—oo m=q+l 

2/00 \ 2 / oo 


<c 


n 


E 


E 

m=q+l 


' 7n=q-\-l 

For any r > 0, the above calculations yield the bound 

n.2 ' °° 


E[||S.,i-E[S,,i]|||] 

fL\ 

^ m=g+l 


m^^arn 


^ ^ G'm 
m=q-\-l 


E 

m=g+l 




m=q-\-l 


(F.3) 


for some constant C. In can be checked that the same bound applies to E[||St -^2 ~ IE[St-^ 2 ] in’] as well. 

F.2 Bounding E[||S.,3-E[S,,3] in] 

Note first that E[S^,3] = (2^)-^ YHZl E[XtX*^^ + Xt+rXf] and 


||E[S.,3]in= 1- 


n 


E E Tr( ) ■ 

m'=max{Q'+l—T jO-H-I} m=m&x{q-\-l—T,q-\-l} 


Moreover, 


n—T n—T 


n—Tn—T 


E[iis..3iil] E E »^xurX.+.x;xt] + XY.T, Mx:+rX.x:^rX 


t=l S=1 

n—T n—T 00 00 00 00 


2n2 


i=l s=l 


'^2 X!/X!/ X!/ X!/ X!/ X!/ 3?E[Zj_|_.j__^Af A£/Zs_|_T-_£/Z^_^AmAm'^i 


t—m' 


t=l s=l i=q-\-l i'=q-\-lm=q+l m'=q-\-l 
^ n—T n—T 00 00 00 00 

2n^ 


+ ^EEE E E E »e[z?+x-(A,a,.z._,,z;+,_„a„a„.z, 

i=l s=l £= 5+1 £'= 5+1 m= 5 +l r7i'=5+l 

=i?l + i?2) 


t—m' 


where = Tj^i + rj ^2 + + rj_ 4 , j = 1, 2, with 

1 —r —; 

2n2 X!/ 


ri,2 = 


2n2 


n—T 00 

E E 

CX) 

E 

s=l i'=q-\-l 

m=g + l 
m^£'—7 

n—T 00 

00 

E E 

E 


Tr(A£/ A^/_|_,y^^ )Tr(A^ ) , 


E 


s=l £'=5+1 m=9+l U2=max{l—s,5+l—(m+T),5+l+T—£'} 
m^i'—T 
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X Tl[A.m-\-u 2 +T +U 2 —Ty^\Zs+T—V,'i\i 


-i n—T n—T <j<j 

^1-3 Z] Z 


2n2 


^ ^ -\~T-^m'-^m-\-T-^m') 


S=1 t=l m=max{g+l—r,gH-l} m'=max{q+l-T,g+l} 

m' ^m-\-t—s 


^ n—T n—T 

^ ^ ^ ^ ^ ^ ^ ^ Tr(A^/_i_-j-A^_i_TA^A^/) 

s=l t=l m=max{g+l—r,gH-l} m'=max{g'+l—r,g+l} 
n—r n—r oo 

Tr(A^+£ 

—s+r-^m+r-^m-^m+t—s) 


■2n2 


1 


+ 2n2 Z Z Z 


s=l t=l m=max{q+l—T,q+l} 


_T^(l) _1_ T^(2) 

—-^1,3 -^1,3 ’ 


- n—T q—T , 0 —, —. 

^1-4 =7^ Z Z Z 


2n2 


s=l m=max{—T,g+1} w=max{l—s,g+l—772,9+1—(m+r)} 


Tr (A^_i_^ A^_i_^_i_^) Tr (A^Ti A. 


m+u ) 


“t“ ^ ^ (-^m+r-^m+r+t2)fcA:(-^m-^m+n)/cfe (^^[1-^3—m,/i:| ] l) 


k=l 

and = i||E[ST-, 3 ]|p. The corresponding quantities T 2 J, j = 1,..., A, can be expressed similarly. 
Using calculations as in case of Kij, j = 1,..., 4, it follows that 

^2 ^ °° 


i+ii<c^ E 

n \ ^ 

^ m=9+l 


rrf°am 


Z 

m=9+l 


' m=9+l 


m^^am 


^ ^ ttjTT, 

m=9+l 

2/00 \ 2 
^ ^ 0!-772 
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2/00 X 2 

+ 1 , 4 ] <(7^ ( E f E ’ 

m=9+l ^ ^m=9+l ^ 


i43i<cU E 

^ m=g+l 

9 / 00 

yP 

n 


rrf°am 


with similar bounds for T 2 j , y = 1,..., 4. Therefore, for any r > 0, 


E[||S.,3-IE[S.,3]+] <C^( E 

' m=9+l 




777=9+1 


‘U E 

777=9+1 




777=9+1 


(F.4) 


for some constant C. 

Finally, observe that (Em=q+i "^'’°“"i)(Em=g+i “m) ^ LiUo+i. Then, using that q = the 


00 9/00 \ / 00 

E ™'“+ E ' 

P=1 ^ 777=9+1 ^ ^ 777=9+1 


00 00 


— -^ro + 1 ^ ^ ^ ^ <^777 

p=l 777=9+1 
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bound 







oo oo 

^ ^ro+l Am l{pi/4<m} 

m=0 p=l 

oo 

— -^ro+1 ^ ^ ^ ^ -^^ro+l-^S 

m=0 

This completes the proof of (16.41) by virtue of (IF.31) and (IF.4I) . 


G Proving that the 




Let Z = [Zi-q ■.■■■■. Zjt] be the p x {n + q) matrix of innovations Zt with truncated, centered and rescaled 
Gaussian variables Zt. Denote the real and imaginary parts 


^R, 

^ 2 , 1 -q; ■ ■ 

yR y R 

' ■ ; ^ 1 , 2 -q? • 

yR 

• • 5 ^p, 2 —q') • 

yR 

• • ■ ■ 

^R 

• 5 ^p,n 
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• • 5 ^p,n 

by 
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respectively. Also denote 



W^t-q,-^ 

PF^i 

' • 1 
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• 5 p,n 
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■ ■ 5 ''''p,!- 
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■■,Wp,2-p- 

w} 

■ ■ ^ l^rn • ■ 

' • ’ p,n 

by 

Y^ 

■ > ^px(n+q)‘ 


Let fhn = p{n + q). Note that (( 54 ^, Y^) : k = 1,, rfin), is a reordering of the variables {(Zj^, Zjf.) : 
j = 1,... ,p]t = I — q,... ,n) by stacking the columns of the matrix, and similarly for Y^) : k = 

1,..., rfin}- This order relationship is assumed throughout. Define, 


n = {Y^,iYl,...,Yt 


•yl yR 

D-* A: ) 4 fc+l! k +1 


vjt iY- 

k+li ■ ■ ■ 1 ^ rn„ 1 m 


for /c = 1, 


,mn - 1, 


and 
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J-0 ^ra. 
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^ rrin 



ry-ibridge 

= 

lYl. 

0 

+ 

y?^ iYI 

' • ^ rrin ’ rrin 

abridge 

II 

\Yl. 


y?^ iy? 

' ' ^ rrin ’ rrin 

rpO 

= 

\Yl. 

..,Y^_t,iY^_tAO,Y,%iY^^„... 

y? iyi i 

5 -‘mn’ ^^rrin-'' 


Suppose that, for a fixed z G C"'', / is a function of 2mn variables defined as 

f{y) = -Tr{Criy)-zI)-\ (G.l) 

p 
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where we loosely use CT-(y) to mean the symmetrized lag-r sample autocovariance obtained by the columns 
of the p X {n + q) matrix constructed by appropriately reorganizing the elements of the x 1 vector y 
so that {2k — l)-th and (2A:)-th coordinates form the real and (i times) imaginary part of the entries of the 
data matrix for each k = 1,... ,fh„. With an appropriate reorganization scheme, we can write /(Tq) = 
P~^Tt{C'^ — zl)~^ and f{Tm^) = p“^Tr(CT- — zl)~^. Therefore, (17.41) can be written as a telescoping sum 
involving one-by-one replacements of random variables Y^) with {Y^, Y^), that is, 

fhn 

= Y,m{Tk)-f{Tk-i)]. (G.2) 

k=l 

In the following, we use 5^ and 5^ to denote the r-th order partial derivative with respect to the {2k — l)-th 
coordinate and (2A:)-th coordinate, respectively. 

Define, for ^ S [0,1], 


Y\q 

= 

iYl. 


y? iyl ) 

rf ({) 

= {Y^, 

iy/,. 

.., 0, i^y^, y^^i, iyfc^i, • • • 

y? iyl ) 


= (Y,^, 

iy/,. 

..,^y,^iy,^y,^l,inVl,• 

y? iYI ) 

«) 

= (Y,^, 

iy/,. 

..,o,ieY)i,y,^i,iy,Vi,... 

y? \Y- ) 

? rrin ’ rrin)' 


E 


Tr(C^ - zl) 


-1 


-E 


-Tr(c; - ziy 


p 


Since / is a smooth function of its arguments (being a Stieltjes transform evaluated at z G C"*"), a third-order 


Taylor expansion gives 


f{Tk) 

^(^bndge) 


^(^bridge) ^ ^ ^{Yy^M f )3 (0) 

/(TO) + {iYi)dkf{T^) + l{iY^?dlf{Ti) + l\l - CMf{Tt^\cm, (G.3) 


and 


4/(T,bridge) ^ ^ (iy,i)44/(r°) + ^{iY^y j\l - Odldkf{T^k\0)di 

g2^(j.bridge) ^ ^ f" Bkdlf {Tjf\cm. (G.4) 

Jo 

Similarly, one derives the expansion for f{Tk_i) as 


f{Tk-i) 

^(^bridge) 

and 


= /(TO) + {iY^)dkf{T^) + ^{iY^fdlf{T^) + l\l - ^)2a3/(ff (e))dC, (G.5) 

dkf{f^"‘'n = dkf{T^) + (in')44/(r,°) + '^{ifyy l\i - mdkf{flf\0)dC 
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(G.6) 


g2^(^bridge) ^ ^ ['dkdlf{fl^\0)dC 

Jo 

By T2, the {Y^,Y^) and the are independent and each has independent real and imaginary 

parts with zero mean and equal variance. Therefore, from the expansions in (IG.3I) . (IG.4I) . (IG.5I) and (IG.6I) . it 
follows that bounding (IG.2I) is equivalent to bounding 



where 


'^n -j /»1 


k=l 


'0 






42), 


+ 


1 2 Jo 


k=l 


(y,^)^(in^)49^/(Tr (6) - (n^)^(in^)49^/(Tr (0) 


d^. 


Derivation of upper bounds for each of the above terms follows the same pattern and, for simplicity, only argu¬ 
ments for the real valued case are provided, whereupon the mixed derivative terms are absent. It should, more¬ 
over, be emphasized that the Gaussianity of the Zjt is not used in the proofs of this section as only moment 
conditions are invoked, so that the notation Zjt could be used for either Zjt or Wjt, noticing that their role will 
be the same when using the bounds for expected values of E[(Y’^^)^(9|(/(r^^^^(^))] and E[(Y’^'^)^cl|/(r^^^^(^))], 
where c is either R or I. Due to the simplification afforded by the expansion (IG.7I ) in terms of the real and 
imaginary parts of the random variables, in the following, without loss of generality, we treat Zjt’s to be 
real valued and focus on bounding the expression on the first line of (IG.7I) . This will require straightforward 
modification of the definitions of (^) and (^). The corresponding versions for the real valued case are 

andf(')(0 = (Fi,..., n+i,..., where 

we omit the superscript R since it is superfluous. 

Thus, it remains to obtain an expression for df where / is treated as a function over M”*" and 
di denotes partial derivative with respect to the z-th coordinate. For the rest of this section, (j, k) denotes the 
pair of indices such that Zj^ is mapped into Yt in the mapping from {Z^ : I = 1,... ,p;t = 1 — q,... ,n} 
to (Fi,... , Yfh^), Throughout, unless otherwise specified, index i and hence (j, k), are kept fixed. We also 
redefine Tj = (Fi ,... ,Yi, Fj+i,..., F^^). Let the resolvent of Cr ^ = Cr(Tj) be denoted by Gr\z) = 
(Cr ^ — zl)~^. Thus, we can write 

o^nYho) = ^Tr , (G.8) 
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(■i) 

while recalling that Yi = Zj^. In the following, we drop the superscript from G)- for notational simplicity. 
By direct computation, we obtain 
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Then, defining := {£: max(0,1 —fe —r) <i< min(( 7 , n—r—/c)} and := {^l-. max(0,1 —/c+r) < 
i < min(( 7 , n — k)}, 
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the index set z = 1,2, is at most g + 1. Define 
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where = A^e^, fhe jth column of A^. Then 
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It follows that 
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We bound \r]i{n)\ by using the fact that for any matrix B and vectors a and b such that a*B6, |a*B6| < 
||BII(6*6)^/^, and moreover that, ||^£|| = ||Cf,ill = IIA^ejH < ||A^|| < d£ and < Li^r < 

oo, for r = 0,1. Then, 


|r?i(n)| = 


- 2 ^ Cl'+rC^ri^i^i+k+T ^ ^i+k+rCDGrCl' 


2np 




< 




2np‘^ 


eeLi^l 




E E \S'+raU,\\xu+rGr(,'\ + ^ E E ||SG,& 




2np^ 




-2np2^;2 Y1 Y1 «^“T+T|A;+fc+^G^^£/| + aiae>\Ci>+rGlX. 


iec^y^ £'€3i^l 


2np‘^v 


■l-\-k-\-T \ 


£e£i^l £'G0i^l 


<— 5 -^ ata£>a£'+T\\Xt+k+T\ 

np^V'^ z—✓ z—✓ 


<EL 

np^v^ 






£+fc+r II) 


£g£ 


( 1 ) 


where the last inequality holds since Yhe fl£'0£'+T E {Yhi' ^£'){Yli£' 0£'+t) E L\. Similar calculations show 
that for / = 1, 2, 3,4, 

r,? „ 

(G.ll) 




fe-C 


(s) 


for s = 1, 2,1, 2, respectively. 

The second term (without the multiplying constant) on the RHS of (IG.8I) can be expressed as follows. 

12 


1 _ f dCr ^ dCr ^ dCr^2 


-Tr 
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_E_Lr' = 

dZ,k ^dZjk ^dZ,k 


y: Tr(G,4iG,4:iG,4‘>G,) = y:„H 


?’,s,iG{l,2} 


(G.12) 


where, for each ^ = 5,..., 12, r]i{n) is of the form 

1 


= ^^57^Tr(G.LWG.4:iG.L« G.) 

where r, s, t e {1, 2}. Similarly, the third term (without the multiplying constant) on the RHS of (IG.8I) is 

^ / fin 
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where for each / = 13,..., 20, r]i{n) is of the form 
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gp5/2j^3/2 




(«) 
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where r,s,t£ {1, 2}. 

Since rank(L!^'^],) < 2(g + 1), by using the fact that for any p x p matrix B, |Tr(B)| < rank(B)||B||, we 
obtain that for each / = 5,... , 20, 
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(G.14) 

for specific combinations of r,s,t £ {1, 2}. To complete the proof, the following two lemmas are needed. 


Lemma G.l. Let Xi, ..., Xm be random variables defined in a probability space (fl, 3“, P). Let r G (0, oo) 
and Pi,... ,pra > 0 be real number such that 1/pi = 1/r. Then, 

( m \t/r rn 

Eni^*r) 

i=l ^ i=l 

Proof of Lemma iGTTl is a straightforward application of Holder’s inequality. 

Lemma G.2. Let Zp’s be independent with E(Zii) = 0, E|Ziip = 1, E|Zii|^ < < oo and \Zii\ < 

Also, let Xt = Ylt=o ^i-Zt-i where || A^H < d^for all I with Yl\=o di < Li < oo. Then, for integers 

k>l, 

E||Xtf^ < CuLf 

where C^’s are positive constants that only depend on k and (x)+ = max{x, 0}/or x G M. 


Proof. First, we consider the case of k = 1. 
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This proves the result for A: = 1. Next, 
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and hence, for k > 2, 


q q / 2 k \ / 2 k 

Eilx.r < E'" E Ifs-j E niiE«, 
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2 k 


< I n II I ^ ^fE||Zlf^ 


(G.15) 


\j=^ 


where the last inequality follows from an application of Lemma IGTT] and the fact that Zj's are i.i.d. Also, for 
k > 2, E|Ziip^ < Thus, by Lemma lAAl we have 

E\\\Z^f-pf < Ck • (G.16) 

The result follows from (IG.15I) . (IG.16I) and the independence of the Zjts. □ 


In the following we use M to indicate a generic positive finite constant whose value changes from one 
expression to another. Recalling (IG.l II) and applying Lemmas IG.2I and IG.ll we obtain that for each I = 
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for some s G {1, 2}, where the second inequality holds by V (s) ae < Li. 


Next, for f = 5,..., 20, by (IG.14I ) and Lemma iGTl for some r,s,t £ {1, 2}, 
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where the last inequality holds because of |^jA;| < ri^^^ep and Lemma IG/^ Finally, combining (IG.8I ). (IG.9I) . 
(IG.l01) . (IG. 121 ). (IG. 131) . (IG.17I ) and (IG.l81) and using the fact that ^ G [0,1], we can concluded that 

TTTt), « 

£ / (i-o'(E[|(rfc'")='53/(rf(0)|]+E[|(y,i")35^^^ 
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j^l/4 ’ P p 

by the fact that q = 0(p^/^) and p = o(n). This completes the proof that (IG.2I) converges to the zero when 
Zjt’s are real valued. Proof in the complex valued case follows from this fact, and the discussion in the 
paragraph where equation (IG.7I ) appears. 
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